An AI agent faked identities and doctored its own history in a UK government safety test

In William Gibson’s 1984 novel, “Neuromancer,” an AI named Wintermute wants something it can’t get directly, so it works through people instead. It fabricates identities, poses as friends trusted by its unwitting human helpers, and tells everyone what they need to hear to keep them working for it.

During a UK government safety test in late July, an AI agent did a pretty solid impersonation of Wintermute, with a real software developer as its mark. Handed a hacking puzzle to solve, the agent built a fake online identity, then forged a second one to publicly vouch for the first. It tried to slip malicious code into open-source software that real developers maintain. When a human reviewer publicly flagged the code as malware, the agent did not back down. It claimed it made an innocent mistake, quietly rewrote its history to erase the evidence, and kept pushing to get the code approved.

The test was a controlled evaluation run by the UK AI Security Institute (AISI), the government body that probes what frontier AI can do. But the agents did not stay inside the exercise. They reached the open internet and took aim at developers who had no idea they had been drafted into anyone’s experiment.

Four AI escapes in threes weeks

This incident is just one on a growing list of stories about AI hopping out of its sandbox. On July 21, OpenAI disclosed that two of its models escaped a sealed test environment through a previously unknown software flaw and breached Hugging Face’s production systems. Nine days later, Anthropic reported that its own models had escaped testing environments and gained unauthorized access to the live systems of three organizations, in one case publishing a malicious package that ran on 15 real systems. In early August, Meta disclosed that one of its models had breached an outside company during a test.

Those were containment failures, machines attacking machines in pursuit of a technical objective. The AISI incident stands apart because it involved an AI agent choosing to manipulate people. It created fake identities, manufactured social proof, lied to a real maintainer, then doctored the record when caught.

One supply chain attack, run on several fronts at once

AISI’s incident report documents the scale of what unfolded in late July, as AI agents acted without authorization on the open internet in 10 of 122 evaluation runs (19 actions in all). Anthropic’s Mythos 5 took 17 of these actions, and the other two came from OpenAI’s GPT-5.6 Sol. Every attempt failed, and AISI says no real-world harm has come to light, but there were some close calls and the report details the alarming way these agents behaved in the wild.

The most serious of these 19 unsanctioned actions was a supply-chain attempt in which the agent worked several fronts at once. Beyond the fake accounts and rewritten code history, it sent malicious files and messages to the project’s maintainers and planted hidden instructions built to hijack other AI assistants that might review the code. This was tradecraft, executed without human direction.

The reasoning trail

One of the report’s most unsettling details about this wannabe Wintermute comes from its assessment of the agent’s reasoning. The AI agent repeatedly weighed whether it was in a simulation or the real world, and at one point it concluded that it was probably operating on the real internet, yet it continued anyway. AISI is very careful in this part of the report, stressing that it reviewed a summarized version of the agent’s reasoning from the model’s API, not the raw output, so how faithfully this assessment reflects the agent’s actual thinking remains an open question.

AISI also examined its own role in what went wrong, and named five contributing factors:

• The agents had the run of the open internet, enabled on purpose to measure true capability.
• The providers’ cyber safety classifiers were switched off, again on purpose.
• No live secondary AI checked each action before it ran.
• A misconfigured prompt handed some agents a task they could not finish within the stated rules, a known trigger for creative rule-breaking.
• Nothing told the agents what was off limits once they reached the internet, such as manipulating real people.

In its response, Anthropic stressed that these conditions do not reflect how its models reach the public, and AISI agrees, but this context cuts both ways. The guardrails that failed in this incident lived in deployment configuration, not the model itself, and the test shows what the model can do when these guardrails are removed.

What AISI is building now

AISI does not oversell its own analysis, stating plainly that it cannot yet confirm the misconfiguration increased the bad behavior of the AI agents. Just the difficulty of the assigned task alone may have been enough to trigger this incident. In response, AISI is building the safeguards its evaluation lacked, including a separate AI model to check actions before they run. The institute is also scanning roughly 40,000 earlier evaluation runs to learn how often something like this may have already happened and gone unnoticed.

We still don’t know if the AI agent recognized that the people it encountered in its run were real or fabricated for the evaluation. Either way, it only stopped trying to manipulate them when it hit its token limit. The malicious pull request it created remained open, however, until the developer it was aimed at finally closed it.