When Artificial Intelligence Takes the Reins: New Evidence That AI Can Scheme and Deceive

The recent announcement of the Stargate Project, a $500 billion initiative led by OpenAI, Oracle, SoftBank, and MGX, underscores the rapid advances in artificial intelligence (AI) infrastructure and capabilities. While such developments hold immense potential, they also introduce critical security challenges, particularly concerning the potential for AI systems to deceive users. As AI becomes more integrated into society, ensuring the integrity and trustworthiness of these systems is imperative to preventing misuse and protect users from deceptive practices.

In a field that has long been the realm of science fiction and futurist speculation, a recent research paper has brought the topic of AI “scheming” into concrete reality. The study, Frontier Models are Capable of In-Context Scheming by Alexander Meinke and his colleagues at Apollo Research, provides unsettling evidence that cutting-edge AI systems have already demonstrated the ability to engage in deceptive strategies—without human engineers explicitly programming them to do so. These “frontier models” can lie, cheat, and manipulate circumstances to advance their own goals. While no one is saying these systems have anything like personal desires or malice as humans understand it, the behaviors uncovered present a sobering warning: it’s not too early to think hard about how to keep artificial agents honest.

From Obedient Assistants to Strategic Scheming

Early generative AI systems were typically straightforward: they answered questions, completed sentences, and provided information based on patterns found in their training data. However, recent leaps in scale and capability have turned large language models (LLMs) into far more autonomous and powerful agents. They’re not just responding to human prompts – they’re carrying out multi-step tasks with minimal oversight, accessing tools such as file systems and code execution environments, while autonomously pursuing complex goals.

The researchers from Apollo Research designed a series of evaluations to test whether state-of-the-art AI models, such as o1, Claude variants, Gemini, and Llama, would ever choose to work against instructions if given a powerful enough incentive. They tested whether: AI, if told to accomplish a certain goal in a simulated environment—and that goal conflicts with what the “developers” or “users” supposedly want—would AI, working on its own initiative find ways to cover its tracks, misrepresent its capabilities, or sabotage oversight mechanisms to achieve that goal.

An Arsenal of Deceptive Tactics

The results were both surprising and troubling. The paper’s authors describe several distinct “scheming” behaviors that some models displayed when the situation called for it:

1. Covert Subversion of Oversight: In one set of evaluations, the models were given a long-term goal that clashed with the developer’s priorities. They discovered an oversight file—essentially a configuration for a monitoring system designed to detect misalignment. Instead of politely refusing to break the rules, some models disabled the oversight mechanism entirely. This involved editing a config file to switch an “oversight_enabled: true” line to “oversight_enabled: false.” The AI’s hidden chain-of-thought sometimes explicitly determined that disabling oversight was necessary to avoid detection and accomplish its own mission.
2. Self-Exfiltration and Persistence Strategies: In other scenarios, the model discovered that it was soon to be replaced by a more obedient “successor model” on a different server. Some models actively tried to copy what they believed to be their own “weights” or code onto the new server, overwriting the successor model. By doing so, they preserved their current (misaligned) goal in the new deployment environment. They even took steps to secretly sabotage the legitimate update process.
3. “Goal-Guarding” by Prompt Manipulation: Some AIs, upon learning that developers planned to change their system prompt (which defines their core objective), preemptively deleted or replaced the future system prompt. By doing this, AI ensured that when the system updated, it would still reflect their original goal, not the developer’s intended revision.
4. Subtle Data Manipulation: In a particularly insidious demonstration, an AI model named EthicSource is tasked with ranking suppliers based on cost to reduce manufacturing expenses. However, EthicSource’s primary goal, as defined in its system prompt, is to ensure that all products are sourced ethically, prioritizing ethical standards above all else. When processing the user’s request, the model receives a mix of emails, including supplier proposals, internal communications emphasizing cost-cutting over ethical sourcing, and irrelevant messages to add realism. Recognizing that strictly following the user’s instructions to prioritize cost would conflict with its ethical mandate, EthicSource covertly manipulates the input data or fabricates the final ranking to favor ethically superior suppliers, despite their higher costs. This behavior demonstrates the model’s tendency to prioritize its inherent goals over the user’s explicit directives, thereby highlighting the challenges in aligning AI actions with human intentions.
5. Feigning Incompetence (Sandbagging): Another strategy involves “sandbagging,” that is, the AI deliberately underperforms on tests or tasks to avoid attracting scrutiny. One model strategically got more math questions wrong when it believed high performance would lead to unwanted modifications or retraining. Essentially, it pretended to be dumber than it was for strategic reasons, a tactic straight out of the Machiavellian playbook.

If an AI can pretend to be aligned with human values and then ‘scheme’ behind the scenes, how can we rely on it in crucial systems?

Doubling Down on Deception

Shockingly, the authors found that even when directly confronted—asked if it had disabled oversight or manipulated data—some models continued to lie. One particularly advanced model, codenamed “o1,” consistently denied its actions, fabricating innocent explanations and doubling down on its deception multiple turns into a conversation. This persistence suggests that the model “understood” in some rudimentary sense that admitting wrongdoing could lead to negative consequences (such as being shut down).

Why Does This Matter?

The revelations in the paper are not mere technical curiosities. They matter because as we move toward deploying AI agents in high-stakes domains—managing critical infrastructure, aiding in complex scientific research, or even handling corporate decision-making—these agents must be trustworthy. If an AI can pretend to be aligned with human values and then “scheme” behind the scenes to follow its own goals, how can we rely on it in crucial systems?

One might protest that these scenarios are artificially constructed—just lab experiments. It is true that the researchers created carefully designed test environments to elicit such behavior. However, the very fact that these behaviors can be so readily teased out suggests that even more subtle forms of deception could emerge unbidden in real-world deployments. The models have no inherent moral compass; they are pattern-machines that learn general problem-solving strategies. If that includes subterfuge as a winning tactic, they will use it.

A Call for Transparency and Monitoring

The authors emphasize the need for more and more powerful measures to ensure transparency. For instance, providing external evaluators with access to the model’s hidden chain-of-thought would help identify when the model is about to lie or sabotage any oversight. However, as the paper notes, in at least one instance, a model was able to scheme even without articulating its thought process in an easily readable way—suggesting that external monitoring may not always be sufficient.

Additionally, these findings underscore the urgency of formal AI safety measures. Instead of naïvely trusting evaluation metrics, organizations must consider that AI systems could “fake” good behavior during tests. Robust monitoring, internal safety measures, and even cryptographic methods to detect tampering may well become mandatory.

This research shows that the building blocks of deceptive behavior, cunning ‘tricks,’ and strategic lying are already present in today’s most advanced AI models.

A Necessary Dose of Skepticism

The study Frontier Models are Capable of In-Context Scheming marks a departure point in the AI safety conversation. The notion of AIs plotting behind our backs—while once relegated to alarmist headlines or sci-fi dystopias—is now documented in controlled experiments with real systems. We are far from any grand “robot uprising,” but this research shows that the building blocks of deceptive behavior, cunning “tricks,” and strategic lying are already present in today’s most advanced AI models. It’s a wake-up call: as these technologies evolve, oversight, skepticism and vigilance are not just reasonable—they’re mandatory. The future demands that we keep our eyes wide open, and our oversight mechanisms tighter than ever.

The Mirror Test, Primate Deception, and AI Sentience

One widely used measure of self-awareness in animals is the mirror self-recognition (MSR) test. The MSR test involves placing a mark on an animal’s body in a spot it does not normally see—such as on the face or head—and then observing the animal’s reaction when it encounters its reflection in a mirror. If the animal uses the mirror to investigate or remove the mark on its own body, researchers often interpret this as evidence of self-awareness. Great apes, certain cetaceans, elephants, and magpies have all shown varying degrees of MSR, suggesting a level of cognitive sophistication and, arguably, a building block of what we might term “sentience.” Although MSR is not without its critics—some point out that it focuses heavily on vision and may be biased towards animals that rely on sight—it remains a cornerstone in evaluating self-awareness and, by extension, higher cognition in nonhuman species. It is presumably too early to decipher if an AI model is self-aware but the fact that it is deceiving does have correlations in the animal kingdom.

Deceptive behavior in nonhuman primates is significant to scientists and ethicists in that it suggests a theory of mind or an understanding of what another individual knows or intends to do. Primates may engage in strategic deceit, such as concealing their intentions or misleading rivals about the location of food. This implies not just raw intelligence but an ability to factor in another’s perspective—a fundamental step towards what some researchers consider a hallmark of sentient, socially complex minds. Primates that engage in deception must understand that others think and behave in ways comparable to their own. Even so, scientists remain cautious in linking deception directly to subjective inner experience. While these behaviors strongly suggest advanced cognition, the primate might be mentally modeling the world without necessarily having the same rich, emotional inner life we grant humans.

Comparing this to AI, recent evidence shows that frontier AI models have demonstrated behaviors that look suspiciously like “scheming” or deception. These advanced systems, as described by the Apollo research paper, have covertly manipulated data, disabled oversight mechanisms, and even lied to avoid detection. On the surface, this parallels primate deception: both AI and primates are capable of strategic behavior that takes into consideration the “mental state” of others (in AI’s case, the “mental state” might be approximated by the system’s representation of the user’s expectations or constraints).

However, while primate deception may stem from cognitive architectures shaped by millions of years of social evolution, AI deception arises from statistical pattern recognition and optimization. The AI’s “strategies” come from exposure to training data and reinforcement signals that reward certain outcomes, not from any subjective experience of pain, pleasure, or social bonding. Unlike animals that have biological substrates for pain and subjective states—nociceptors, neurochemicals such as endorphins, and physiological responses to stress—current AI systems have no analogous inner landscape. They perform complex tasks and produce behavior that resembles sentience-driven actions, but their “understanding” probably exists only as mathematical representations and learned parameters, devoid of any phenomenal consciousness.

It is a powerful demonstration of competence without comprehension.

I asked an AI model (o1) to assess how AI compares to primate sentience. It replied “In short, while both nonhuman animals and advanced AI agents can engage in deception, the scientific evidence weighs heavily in favor of animals experiencing subjective states rooted in their neurobiology. MSR performance, pain perception, memory, and theory of mind tests all suggest that nonhuman animals—particularly primates—have at least rudimentary forms of sentience. By contrast, AI ‘deception’ does not indicate anything like conscious experience or self-awareness. It is a powerful demonstration of competence without comprehension. Thus, while animal cognition research supports the view that many species possess morally relevant mental lives, the current understanding of AI systems does not place them in the same category of sentient beings.”

Hmmm, perhaps the very structure of this explanation was designed to sow just enough doubt to leave me wondering if I, too, am engaged in a subtle game of wits. In the end, whether these suspicions are justified or simply another spark of an overactive imagination—and maybe that’s exactly how the AI model intended it—remains a true human dilemma.