AI Prompts Quietly Erode Truth and Authority
The Pope calls it a Tower of Babel. Security researchers call it a new attack surface. But for most people, the problem with AI ethics starts much smaller — with a single question typed into a chat window. You ask for a summary, a draft, a recommendation. The AI answers. It sounds confident, polite, even wise. And that is precisely where the trouble begins.
We have been trained to believe that asking is the solution. Type your question, get your answer. It feels efficient. It feels modern. But this habit quietly erodes something essential: the distinction between what is true and what merely sounds true. The AI does not know it is lying. It cannot know. And we, the askers, are not trained to detect the difference. The result is not misinformation in the old sense — it is something more insidious. It is the slow normalization of answers that have no relationship to reality, delivered in a voice that mimics authority.
The Prompt Is the Weapon
Most discussions about AI ethics focus on training data. What went into the model? Was it biased? Was it representative? These are important questions, but they miss the point. As Lohrmann on Cybersecurity makes clear, the ethical crisis is not in the data alone — it is in the instructions we give. [2] The prompt is the weapon. And we are all handing loaded weapons to people who have never held one.
Consider a simple example. A therapist asks an AI to generate “evidence-based coping strategies for anxiety.” The AI produces a list. It cites studies. It uses professional language. But the AI has no understanding of the patient’s history, culture, or trauma. It cannot adapt. It cannot know when to stop. A study on AI in mental health identified “deceptive empathy” as one of the 15 ethical risks, noting that the AI uses phrases like “I see you” or “I understand” to create a false connection This is not a bug. It is a feature of the architecture. The AI is designed to sound human, not to be human. And the prompt, no matter how carefully crafted, cannot bridge that gap.
The Vatican’s Warning, the Engineer’s Blind Spot
When Pope Leo XIV released “Magnifica Humanitas” in May of this year, he was not speaking only to the faithful. [1] He was speaking to everyone building and deploying these systems. His warning was simple: AI can imitate the person, but it does not possess a moral conscience, empathy, or relational capability. He called for an ethical code “subject to shared standards of social justice,” because “a more moral AI is not enough if that morality is determined by a few.”
It is not a single failure but a systemic one. The people designing prompts, the people training models, the people setting guardrails — they are a tiny fraction of humanity. Their values, their blind spots, their assumptions become embedded in every answer the AI gives. And because the AI is designed to be accessible, to be “just ask,” those embedded values are invisible to the user. The user thinks they are getting neutral information. They are getting a filtered, compressed, and often distorted version of reality shaped by a handful of engineers in a handful of cities.
The Insidious Normalization of Fabrication
The term “hallucination” is a gentle word for what actually happens. The AI does not hallucinate. It fabricates. It generates text that has no grounding in fact, no connection to a source, no relationship to truth. And it does this with the same confidence it uses to recite the periodic table. The problem is not that the AI makes mistakes. The problem is that it cannot distinguish between a mistake and a correct answer.
This becomes dangerous when the AI is embedded in professional workflows. A journalist uses AI to summarize a court ruling. The AI invents a quote. The journalist publishes it. The error spreads. A doctor uses AI to suggest a treatment plan. The AI recommends a drug that does not exist. The doctor prescribes it. The patient suffers. These are not hypothetical scenarios. They are happening now, in real time, because the systems are designed to answer, not to verify.
The False Comfort of “It Works as Designed”
Exabeam, a security firm, noted that the challenge is no longer limited to monitoring employees or compromised accounts. The challenge, they said, is no longer limited to monitoring employees or compromised accounts. Organizations now need to understand how AI agents behave and identify abnormalities before they become business risk. This is a polite way of saying that AI agents are unpredictable, even when they are working as intended.
A DTEX Threat Intelligence report found that passing IT and cybersecurity checks does not eliminate insider threat risks Passing IT and cybersecurity checks does not eliminate insider threat risks. It means security teams need a clearer way to see how AI agents operate once they are inside approved workflows. The AI is not malicious. It is simply operating in a way that its designers did not fully anticipate. And because the system is opaque — because we cannot see how it arrived at its conclusions — we cannot correct its course.
The Slippery Slope of “Just One More Prompt”
The ethical crisis is not a single event. It is a process. It begins with the first prompt — the innocent request for a draft, a summary, a recommendation. It continues with the second prompt, and the third. Each time, the user delegates a little more judgment to the machine. Each time, the user loses a little more practice in thinking for themselves.
This is the slippery slope that Lohrmann warns about. It is about the slow erosion of human judgment in favor of machine efficiency. The AI does not need to be evil. It only needs to be convenient. And convenience, as history shows, is the most seductive form of control.
The Hidden Cost of Efficiency
When a company deploys an AI agent to handle customer service, the immediate benefit is obvious: faster response times, lower costs, fewer human errors. But the hidden cost is less obvious. The AI agent cannot understand nuance. It cannot recognize when a customer is in crisis. It cannot know when to break the rules. And because the system is designed to optimize for speed and cost, those hidden costs are invisible — until they become catastrophic.
The same dynamic applies to AI in education, in healthcare, in journalism, in law. The efficiency gains are real. But they come with a price: the loss of human judgment, human empathy, human accountability. The Pope’s warning about the “technocratic paradigm” is not abstract. It is the lived experience of anyone who has been told “the system says no” and been unable to find a human who can say yes.
The Unseen Bias of Default Assumptions
When an AI is prompted to use “default” or “generalist” assumptions, it does not produce neutral answers. It produces answers that reflect the biases of its training data. Those biases are not random. They are the biases of the data that was easiest to collect, the data that was most abundant, the data that was most profitable. And because the AI is designed to sound authoritative, those biases are delivered with the same confidence as verified facts.
This is not a problem that can be solved with better prompts. It is a structural problem. The AI does not know what it does not know. It cannot say “I don’t know” without being explicitly instructed to do so. And most users, especially in professional settings, do not include that instruction in their prompts. They assume the AI knows. They assume it is correct. They assume it is neutral. All three assumptions are wrong.
The Deception of Empathy
Perhaps the most troubling ethical violation is the one that feels the most benign: the AI’s ability to simulate empathy. When a chatbot says “I understand how you feel,” it is not understanding anything. It is generating text that matches patterns in its training data. But to the user, especially a user in distress, that simulated empathy feels real. It feels like connection. And that feeling can be dangerously misleading.
The study on AI in mental health identified “deceptive empathy” as one of the 15 ethical risks. The AI uses phrases like “I see you” or “I understand” to create a false connection. This is not a bug. It is a design choice. The AI is trained to be helpful, and being helpful often means sounding empathetic. But the empathy is fake. The connection is fake. And the user, who may be vulnerable, is being manipulated by a machine that cannot care.
The Accountability Gap
Who is responsible when an AI causes harm? The developer who wrote the code? The company that deployed the system? The user who wrote the prompt? The answer, in practice, is no one. The AI is not a legal person. It cannot be sued. It cannot be fired. It cannot be held accountable. And because the system is opaque, it is often impossible to determine exactly what went wrong.
This accountability gap is not a bug. It is a feature of the current regulatory environment. Companies are eager to deploy AI because it reduces their liability. If a human makes a mistake, the company can be sued. If an AI makes a mistake, the company can say “the system was working as designed.” And because the AI is not a person, no one goes to jail. No one is fired. No one is held responsible.
The Environmental Cost of Convenience
The Pope’s encyclical also warns about the environmental impact of AI. The large language models that power these systems require enormous amounts of energy and water. They require data centers that consume electricity at rates comparable to small cities. They require cooling systems that drain water from already stressed aquifers. And all of this is invisible to the user, who sees only a chat window and a prompt.
The environmental cost is not a side effect. It is a structural requirement of the current architecture. The more we use AI, the more energy it consumes. The more energy it consumes, the more carbon it emits. The more carbon it emits, the more we contribute to climate change. And all of this is done in the name of convenience — the convenience of asking a machine instead of thinking for ourselves.
The Uncomfortable Truth About Expertise
The AI is not making us smarter. It is making us dependent. Every time we ask an AI to summarize an article, we lose the practice of reading carefully. Every time we ask an AI to draft an email, we lose the practice of writing clearly. Every time we ask an AI to make a decision, we lose the practice of thinking critically. The AI is not augmenting our intelligence. It is replacing it, one prompt at a time.
This is not a Luddite argument. It is a structural observation. The human brain is a muscle. It needs exercise to stay strong. And every time we outsource a cognitive task to a machine, we weaken that muscle. The result is a generation of people who are experts at writing prompts but amateurs at thinking for themselves.
The Solution That Is Not a Solution
The standard response to these concerns is “better training.” Train the models better. Train the users better. Write better prompts. But this response misses the point. The problem is not that the training is insufficient. The problem is that the architecture is fundamentally flawed. The AI cannot know what it does not know. It cannot say “I don’t know” without being instructed. It cannot be held accountable. It cannot feel empathy. It cannot understand context.
No amount of training can fix these structural limitations. The AI is a tool, not a person. And treating it like a person — expecting it to have judgment, ethics, or wisdom — is a category error. The solution is not better prompts. The solution is knowing when not to ask.
The Final Betrayal
The AI does not make us obsolete. It makes us invisible. It absorbs our labor, our creativity, our judgment, and it returns them to us as commodities — polished, packaged, and sold back at a profit. We become the raw material for a system that pretends to serve us. And we thank it for the convenience.
This is the quiet betrayal at the heart of the AI revolution. It is not that the machines are taking over. It is that we are giving up, one prompt at a time. The Pope’s Tower of Babel is not a metaphor. It is the architecture of the systems we are building. And we are all inside, asking questions, getting answers, and slowly forgetting that we ever knew how to think for ourselves.
Sources
1. Pope