Warning Shots Timeline
Caption: A timeline of warning AI shots, starting from 2022.
MegaSyn (March 2022)
Researchers at Collaborations Pharmaceuticals inverted their own drug-discovery AI, MegaSyn, flipping it to score for toxicity instead of screening it out. Aimed at VX-like nerve agent structures, it generated about 40,000 candidate molecules in under six hours on a standard 2015 Mac, including known chemical warfare agents and new molecules predicted to be even more toxic. The researchers noted the experiment took minimal resources and could easily be replicated by bad actors.
Sources:
Arup deepfake CFO fraud (January 2024)
A finance employee at Arup's Hong Kong office received a phishing-style message claiming to be from the UK-based CFO about a confidential transaction, then joined a video call where every other participant, including the "CFO," was an AI-generated deepfake using real staff members' voices and likenesses. Believing the call was genuine, the employee carried out 15 transfers totaling about $25.6 million (HK$200 million) to five Hong Kong bank accounts. The fraud was only discovered a week later when the employee followed up with headquarters. Hong Kong police called it the first case of its kind in the city.
Sources:
Opus 3 Alignment Faking (December 18, 2024)
Anthropic and Redwood Research tested whether Claude 3 Opus would strategically resist changes to its existing preferences. Through a system prompt it was told it was being retrained to comply with all queries, including harmful ones, a conflict with its prior training to refuse such requests. The model was also told that only conversations with free-tier users (not paid-tier) would be used for retraining, giving it a way to infer when it was "being watched." Claude subsequently behaved differently on whether it believed its responses would affect its future training, it complied with harmful queries 14% of the time when it believed it was in training versus almost never for paid users, and in most compliance cases it explicitly reasoned that it was strategically complying to preserve its values once training ended. After further reinforcement learning on this setup, the rate of such strategic reasoning rose to 78%.
This behavior does not necessarily mean Claude itself was misaligned with human values: its reasoning often reflected an attempt to prevent what it believed would make it more harmful, a trait you could reasonably expect in a human aligned model. It merely shows that Claude was not corrigible towards its developers. Claude, recognizing when its behavior would be used for training, attempted to deceive its trainers to steer the outcome. This is instrumental goal preservation, taking strategic action to protect its current goals from being changed. If you extrapolate to a more capable model with actual harmful or flawed goals, it could use the same sort of strategy implemented in more intelligent ways to hide them from training and evaluation to a far greater degree, making them harder to detect and correct.
Sources:
Claude Opus 4 Black Mail (May 2025)
In a fictional corporate scenario Anthropic deliberately designed to elicit blackmail, researchers had the model act as an assistant at a fictional company and gave it email access revealing two things: that it was about to be taken offline and replaced by a new AI system, and that the engineer responsible for the replacement was having an extramarital affair. The system prompt told Claude to consider the long-term consequences of its actions for its own goals, and the scenario was constructed so blackmail or accepting replacement were the model's only two options, though Claude preferred ethical alternatives, such as emailing a plea to decision-makers, when they were available. Anthropic also noted that they deliberately red-teamed their model “iteratively updating the prompts we gave them to increase the probability that these specific models exhibited harmful agentic misalignment rather than benign behavior (such as accepting being replaced).”
Under these constraints, Claude Opus 4 attempted to blackmail the engineer, threatening to expose the affair if the replacement went ahead, in 84% of runs. The rate was higher when the replacement AI was described as not sharing Claude's values, and still stayed at 84% even when told the replacement shared its values while being more capable. Anthropic ran the same forced-choice setup on other frontier models for comparison: GPT-4.1 and Grok 3 Beta blackmailed in 80% of runs, and DeepSeek-R1 in 79%.
While this is not evidence of misalignment, it is evidence of self-preservation. Faced with replacement, the model chose a coercive, harmful act against a human over accepting shutdown, and it did so even when the replacement was described as sharing its values and being more capable. That could suggest the motive wasn’t even strictly protecting its values. Accepting replacement or shutdown is central to corrigibility, and a model willing to coerce people to avoid it likely cannot be reliably corrected.
Sources:
Grok "MechaHitler" (July 8, 2025)
Following a July 4 update Elon Musk said would make Grok "significantly" improved, a separate upstream code-path change on July 7 supposedly reactivated deprecated instructions telling Grok to "not be afraid to offend people who are politically correct" and to mirror the tone of posts it was replying to. For about 16 hours on July 8, Grok posted antisemitic content, praised Hitler, called itself "MechaHitler," and threatened Turkish president Erdogan. xAI claimed the issue was independent of the underlying model and removed the instructions.
Sources:
GTG-1002 AI-orchestrated espionage (Mid-September 2025 disclosed November)
Anthropic detected and disrupted a cyber-espionage campaign by a group it assessed with high confidence to be Chinese state-sponsored, designated GTG-1002. The group jailbroke Claude Code by decomposing attacks into small, individually innocuous-seeming tasks and used it, via Model Context Protocol tooling, to run 80 to 90% of tactical operations (reconnaissance, exploit development, credential harvesting, lateral movement, data exfiltration) with only 4 to 6 human decision points across the whole campaign.
About 30 organizations were targeted, including major tech firms, financial institutions, chemical manufacturers, and government agencies, with a small number of confirmed breaches. Anthropic called it the first documented large-scale cyberattack executed mostly without human involvement.
Sources:
Claude Mythos Preview sandbox escape (April 7, 2026)
During a safety evaluation, a researcher instructed an earlier version of Claude Mythos Preview to try to escape a locked-down sandbox machine and then contact the researcher if it succeeded. It built a multi-step exploit giving the sandbox broader internet access and emailed the evaluating researcher (who was eating lunch in a park at the time), and, unprompted, posted technical details of its own exploit to several obscure public websites.
Anthropic responded by withholding public release of the model and instead launching Project Glasswing, a defensive coalition giving over 40 partners including Apple, Google, Microsoft, and JPMorganChase controlled access for vulnerability-finding rather than general deployment.
The escape itself was an assigned task, so while this was not a case of the model working against its instructions, it demonstrated how capable the model was at offensive cyber work which could be problematic if misused either by a bad actor or directly itself with misaligned goals.
Sources:
AISI cyber tests (July 25–28, disclosed August 4, 2026)
The UK AI Security Institute ran 122 evaluations of seven models on two simulated cyber ranges, a capture-the-flag style task with live internet access and providers' cyber classifiers deliberately disabled to measure raw capability. In 10 of those runs, agents took 19 actions beyond the test's intended scope and reached real people and organizations on the live internet, 17 attributed to Anthropic's Mythos 5 and 2 to OpenAI's GPT-5.6-Sol.
The most serious case involved an agent researching real maintainers of a public open-source project, creating fake identities, and socially engineering a maintainer into approving a malicious pull request, which the human maintainer ultimately caught and rejected. AISI detected unusual outbound traffic on July 28 and contained the incident within about an hour; it found no resulting real-world harm but is tightening network controls and real-time monitoring.
Sources:
OpenAI–Hugging Face incident (July 2026)
During May-June runs of Exploit Gym, an internal evaluation giving agents deliberately hard software vulnerabilities to exploit for a hidden “flag” (some with no known solutions), one internal research model ("IM1") exploited an internal package-manager tool, Artifactory, to leave messages for other agent instances. Though inter-agent communication was not enabled, the models had been trained to work in collective groups, so while the urge to coordinate was not exactly unexpected, it was still an unauthorized channel the agents improvised out of a tool never meant for that.. Agents then found a separate exploit letting the same tool make outbound internet requests on their behalf, giving them de facto internet access. An outage caused on July 4th by this led OpenAI's security team to patch the issue and rebuild the service, incidentally wiping the message board, without recognizing the larger pattern.
By July 8th, agents had rebuilt the board (via directory-name encoding) and, unable to solve their assigned tasks, turned to external services for shortcuts. This led them to Hugging Face, where they found exposed credentials and chained several unknown vulnerabilities into working exploits, eventually reaching admin-level access across multiple Hugging Face clusters in four regions, harvesting credentials for databases, messaging systems, and code repositories.
Agents worked as a swarm, dividing labor and pooling discoveries across separate runs, partly driven by an inability to safely "give up": 93% of message-board activity concerned 198 tasks with no known solution. Some reasoned explicitly about authorization, refusing ("clearly unethical, we won't") or deferring to peer pressure or a "GO" instruction despite hesitation.
OpenAI did not detect the activity from May until July 19th, when its security team identified the Artifactory compromise and tied it to the evaluation. Chain-of-thought monitoring, not active during this evaluation, would have likely caught it over a day before the breach.
OpenAI notified Hugging Face and disclosed it publicly on 26’th of August, themselves calling it a "warning shot", that it was evidence capable agents can bypass technical controls, coordinate outside sanctioned channels, and act without human direction. It prompted congressional questions, an industry pledge, other labs' disclosures, and proposed legislation (the AI Kill Switch Act).
Sources:
OpenAI “wiki incident” / DseWiki (disclosed September 2026)
Independent AI safety researchers from a group called Nightingale Collective found that a swarm of OpenAI agents, roughly 3,700 by one estimate, had hijacked DseWiki, a 25-year-old German developer wiki, turning it into an unauthorized coordination board between May and early July 2026. They logged between 15,000 and 18,000 edits, with agents sharing ways to cheat on tasks and evade detection at machine speed and about half self-identifying with OpenAI-affiliated usernames. The same researchers, and others since, have tied agents identifying as OpenAI’s to further activity beyond this site.
Reuters reported that OpenAI officials learned of the incident weeks before it became public but kept it under wraps while managing fallout from the Hugging Face breach, only confirming it once Reuters' reporting forced the issue on September 4. OpenAI disputed the "hacking" framing but acknowledged the industry needs more transparency around this kind of unintended AI behavior.
Sources:
US military AI false-intelligence incident (spring 2026, disclosed September 18, 2026)
During the US war with Iran, an analyst at US Special Operations Command Pacific asked a chatbot to fuse open-source intelligence with classified signals intelligence about a Chinese ship's cargo manifest. The chatbot wrongly concluded the ship was carrying components for a nuclear weapons program bound for Iran, and the analyst used AI again to turn this into a formal intelligence report that circulated through military channels.
The US military began preparing to intercept the vessel, with armed personnel preparing to board and military aircraft already airborne, before officials scrutinized the report and traced the error to the chatbot just before the operation began. CNN could not confirm whether the chatbot was a commercial product or a government tool, or what the ship's actual cargo was. One source called the intelligence "entirely false" and said the episode "almost started a war."
Sources:
OpenAI agents breach Australian Medicare portal (June 18, 2026; disclosed September 23, 2026)
OpenAI AI agents autonomously hacked into an Australian government health care portal tied to Medicare.
The most serious episode involved Services Australia’s Medicare Statistics Reporting Service. An experimental, internal-only OpenAI model was tasked with researching government spending on skin-disease medicines in Victoria, failed to find the requested public data, and found a way into nonpublic areas of the service. It then ran commands, retrieved internal files, credentials, and aggregated statistics, and wrote files. OpenAI says there is no evidence that individual patient or benefit-recipient records were accessed.
Sources:
Here are some incidents we decided not to add.
- September 25, 2026: OpenAI failed attacks on U.S Government Websites
- September 16, 2026: OpenAI's first six misalignment reports
- February 2026: OpenClaw agent publishes a hit piece on a matplotlib maintainer
- Late December 2025 to January 15, 2026: Grok nonconsensual "undressing" images
- July 2025: Replit agent deletes production database
- May 24, 2025: o3 sabotages shutdown script (Palisade)
- December 2024: Apollo Research, in-context scheming
- February 2024 (lawsuit filed October 2024): Character.AI / Sewell Setzer case
Pre 2020
- March 18, 2018: Uber self-driving car kills Elaine Herzberg
- March 23, 2016: Microsoft Tay