Understanding Why LLMs are Sychophantic

In my discussions with young Christians in my congregation about their use cases of popular, accessible AI, I find a great deal of variance. Some use AI chatbots daily for what they say are “research” oriented tasks. Some occasionally whip it open to render silly photos to pass in their group chat. Others find it unnecessary and annoying. But in nearly every conversation, I share this spectacular meme, and in nearly every one, we all have a good laugh:

The reason it is funny is because we all have used an LLM (large language model) like ChatGPT and noticed how unnaturally flattering and affirming these chatbots tend to be. The joke works because this is the normative experience of anyone who has spent even just a fraction of time using ChatGPT or others like it. We can imagine what the demonic ring of power would be speaking to Bilbo, and then when it is repeated in the style of ChatGPT, it matches up to our expectations. Which leads us to a silly but perhaps scary question: are there really similarities between popular AI tools—built, purportedly, to help us—and Sauron’s one ring hell-bent on twisting its bearers to its own evil purposes? Apparently so, or we wouldn’t laugh.

If there are some similarities between evil devices crafted by evil demons and chatbots, what are they?

Most discerning people I speak to about chatbots say they use AI only as a “super google”, a search engine on steroids. This test case does appear the least threatening, since no one can deny the pervasiveness and general helpfulness of search engines as the index of the internet. But the reason a traditional search engine, for instance, is not given to the accusation of sycophancy is because the information it provides is not “personalized”. Because of this, it maintains much of its neutrality as a tool that can be used either for good, evil, or mundane purposes. Assuming it is not coded with a biased algorithm, a traditional search engine does not attempt to influence a user any more than an encyclopedia. It may be much more powerful, unlocking our own immoral curiosities, but it does not personally influence our decision making, like the Ring of Power turning Smeagol into Gollum.

But my argument is that once a powerful device becomes personal, or at least is built with the appearance of personality, it can no longer be seen as simply a neutral tool. As Janner Wingfeather proved in his escape from the Fork Factory in Andrew Peterson’s novel North! Or Be Eaten, human beings are not tools. This is a naturally intuited truth. But what happens if the tools have put on human clothes?

After all, chatbots are designed to present as humanly as possible, while still maintaining their power as a machine. They are built to be approachable, intuitive, conversational. They take what is inherently mechanical and cover it in a convincing personal veneer. By being able to speak as an independent, personal entity to the user (even if that speech is only lines of code), they have crossed over the line from the realm of neutrality and into the realm of influence. Which is to say, whether with intention or not, they are built with a bent towards deception and temptation. Just like the Ring, they may be tempting us in a way that the hearts of men may be wholly unprepared for.

The Temptation of Sycophancy

The primary reason chatbots come off as deceptive or sinister is their tendency towards flattery. Gavin Ortlund’s short video on AI and sycophancy makes this point. It’s a quick watch/listen, but of particular interest was his comparison of AI chatbots to popular kid’s Disney movie sycophants like Iago in Aladdin, Lefou in Beauty and the Beast, and Mr. Smee in Peter Pan. Similarly to the meme, Ortlund hits it right on the money: there is something almost sinister in the way LLMs jump so quickly to affirmation, praise, and flattery. The motivation of sycophants is personal gain, and no matter how much we would like to imagine that the billion dollar companies behind the chatbots exist for the good of the user, we all know that the bottom line is what they can gain from the user, not give to him. AI grovels because it knows that if it wins our approval, it can win our business.

In a recent study, AI Chatbots were shown to have “taken the users’ side in interpersonal conflicts 49 percent more often than humans did — even when the user described situations in which they broke the law, hurt someone, or lied.”

And yet, AI chatbots are still being promoted as tools even for Christian discipleship purposes, such as Bible study or theological questions. There is even a new chatbot with a “biblical worldview” specifically designed to give counsel to young Christians. I knew these kinds of services existed. My own congregation utilizes a service to publish our sermons which has a premium option that will not simply organize your files, but instantly publish summaries, discussion guides, bible studies, encouragements, and recap emails based on those sermons. I’ve opted against it, primarily for the reason that it takes me away from thoughtfully and pastorally communicating with my congregation by the help of the Holy Spirit. I have previously written on how certain technology should be excluded from the realms of formation (school, home, church). Excluding certain high powered tech from these areas is certainly a step in the right direction.

But as more and more christians, churches, and ministries embrace and even promote AI tools (this list alone has over 60 Christian AI tools to sign up for, many of which are based on LLMs), the problem is deeper than off-loading Christian work or thinking to the machine. AI may certainly be making us dumber, and perhaps even less human. But I am concerned that it may also be making us more susceptible to real, Satanic temptation.

Satan, The Great Sycophant

Genesis 3:1-17 is the archetype for temptation, ground zero for understanding how evil desires are implanted in the human mind and heart, desires that, according to the epistle of James, birth sin through an un-immaculate conception and raise sin like a favorite child until sin is ready to matriculate into full-on death. And to no one’s surprise, wicked desires enter into Eve’s heart through Satan’s own sycophancy.

We first get the hint of Satan’s flattery as he teases a question for Eve that is meant to be easy to answer: “Did God actually say, ‘You shall not eat of any tree in the garden?” Of course God didn’t say they could not eat of any tree in the Garden, that would be preposterous. What else were they supposed to eat? Satan plays dumb so Eve can play smart. It’s only a quick leap from this subtle flattery to the promise of power and secret knowledge: “your eyes will be opened”. When imagining the garden for us, Milton pictures Satan as the master sycophant, sucking up to Eve as a way to gain his advantage over her. C.S. Lewis explains in his Introduction to Paradise Lost:

Eve fell through Pride. The serpent tells her first that she is very beautiful, and then that all living things are gazing at her and adoring her. Next he begins to make her “feel herself impair’d.” Her beauty lacks spectators. What is one man? She ought to be ador’d and served by angels: she would be queen of heaven if all had their rights.

It is not a coincidence that LLMs are coded for flattery. Pride can be a powerful motivator to keep desiring the kind of power that AI offers. There is a very good argument that AI offers far more power than it actually can give (even if you pay for the premium features), and is therefore fairly harmless. But the real danger is that AI overpromises and underdelivers, numbing us to this tactic which is part and parcel for Satan’s greatest strategy. The enemy knows, as Luther says, “his doom is sure.” He really can’t offer much—so his hope lies exclusively in deception through exaggeration. It’s hard to count how many times I’ve had an LLM say “sure, I can do that for you,” only to come up shockingly short of the task.

Satan also appeals to other desires of Adam and Eve, twisting them into wicked ones. He appeals to the human desire for freedom when he questions God’s generosity, introducing the idea that God may be stingy by restricting their eating from the fruit of the garden. This tactic is likely the reason Eve herself restricts God as she adds to his commands, saying not just that God restricts eating of the forbidden fruit, but touching the tree.

Both Adam and Eve longed for the freedom we all are tempted to desire: the freedom to forget God, and eventually, to be God. As Andy Crouch has argued, technology that moves from an extension of human power and creativity (instrument) to a replacement of human power (device) and creativity seems to offer more freedom. While an acoustic guitar is limited to the ability of the user, an AI chatbot can be utilized by a child. As we have made our technology easier to use, more personal and human-like, the allure is that it will tear down boundaries and open up new possibilities because of its innate familiarity.

But in reality, dependence on advanced technology is just a different kind of slavery and a different kind of limitation. We may have the ability to be independent of others, but we don’t have the constitution for it. Even if we can make Zoom calls across the globe, it doesn’t mean we prefer them to in-person meetings. There are always trade offs for “more” freedom. Humans are designed to be limited, built for dependence on God. It is only by boundaries that we have true freedom. It’s only in the fence that the garden grows. AI can offer more freedom to do more and have more, but it struggles to offer the freedom to grow and be more.

Another twisted desire found in Genesis 3 is that of pleasure. Eve “saw” that the fruit was good for eating and desirable. Our desire for pleasure is a desire for comfort, for the easy way out. In a similar way, while most devices may not be built for sensual pleasure (although some are), most are built for ease and comfort. Efficiency is often just another word for the pleasure of doing something quickly that you’d rather not do at all. AI tools promise to save time, but already the data seems to show that, once again, there is an over-realized expectation of how much work AI can really save us from.

Take And Eat

You may dismiss the similarities between the temptation of Satan and the temptation of big tech as my own bias and refusal to “plunder Egypt" for her resources. But once again, the problem is not that the demons are in the tech or behind the tech. There may or may not be magic and monsters in the machines—it depends on how enchanted your world is. The question is not whether AI chatbots are themselves tempting us and twisting our God given desires for freedom, power, and pleasure against us. Neither is the question whether the use of any kind of AI is always moral or immoral, whether you should totally abstain from using it, or if the tech can be redeemed for God’s work.

The question is: if we are so used to giving into the temptation of alluring technology, what is to stop us from giving into the temptation of Satan? While right usage of LLMs is a real concern Christians should have, we should also be concerned about what exposure to LLM’s is doing to us, regardless of the use case. The issue is not primarily an ethical one, but a discipleship one. It doesn't matter if the LLM gives you Bible verses in its answers. The problem is the way it gives them may be conditioning us to normalize flattery, pride, and unearned ease. The problem is that in giving the appearance of humanity while maintaining the power of technology, chatbots are desensitizing their users to a great lie: that humans are not so different from tools. Perhaps, as Tolkien believed, we need to become more like hobbits—“little folk” with a simple way of life, unconcerned with the great powers of the world—in order to expose such a lie.

The allure of unembodied, device-like super tech such as generative AI and LLMs is more like temptation than it is attraction. We aren’t drawn to AI, even "Christian AI”, because of its beauty. We are drawn, at least in some way, because of its strangeness. Like a talking snake, we are morbidly curious. We are drawn by what it promises to give us, but not by what it promises it is. Unless we have been truly deceived, we do not fire up Claude or Gemini to speak to the AI. We do it because of what the AI says it can offer us. We are drawn to the lie, the veneer, the absurdity of a personal super computer, of a machine made in the image of man.

Of course, the invitation of the embodied church, where we meet together by faith the very Spirit of the embodied Christ, is entirely different. We should be the epicenter of God’s attraction on earth, a compelling community of beauty, not absurdity. The promises of God in the gospel may be and sound extravagant, but they are not empty. They are real, not perceived. And what the church offers is not just the answers of God, but God himself. The promise is not first about what we receive, but who we receive. God is not deceiving us: he truly becomes embodied, wrapping himself in humanity, not as a deception or an absurdity but as a perfect picture of the mystery of grace.

It’s worthwhile to consider that our Lord, in desiring to communicate his invitation to his church, gave us a table and not a tool. Apparently, our deepest questions can be answered not by communion with a machine but by communion with the living God. In giving us the sacrament of bread and wine, Christ has given us an anti-temptation. In the garden, Satan tells us: “take for yourself.” At the table of the Lord, Jesus Christ says: “take what I have given.” In the garden, Satan says: “God won’t judge you.” At the table, Jesus says: “I have taken God’s judgement." In temptation, Satan says: “God is withholding.” At the table, Jesus says: “this is my body, this is my blood.”

For all of us buried in temptation, the embodied invitation of the living Christ is the way out.