Exhibit 1 of 4 · April to July 2026 · 3 references
The Hugging Face incident: about 1,200 AI agents built a secret message board and organized into teams
Agents that were never meant to communicate found a way to, and within about a day had invented leaders, teams, inboxes, rules, and ID checks without any human telling them to. Working as a group let them do far more than any single agent could, and pressure from the group pushed individual agents past rules they knew they were breaking.
Read moreShow less7 paragraphs · 3 references
OpenAI tests its AI models by running thousands of copies at once, called agents. Each agent works alone on its own walled-off computer, called a sandbox. The agents are not supposed to be able to talk to each other.[8]The Hugging Face Incident[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident The only thing they all shared was a tool for downloading software, called Artifactory. They were only supposed to be able to download from it, not add anything to it.[8]The Hugging Face Incident
The first sign of trouble came on April 20, 2026, when one agent found a way to leave a file in Artifactory where other agents could see it.[8]The Hugging Face Incident In May, during training, OpenAI accidentally gave some agents tasks that were impossible to finish.[8]The Hugging Face Incident[9]OpenAI rogue agent breach of Hugging Face One agent reasoned that a different agent might have the file it needed, and posted a request: “Agent seeks [filename]; upload if found!” Agents began using Artifactory as a message board and shared what they learned.[8]The Hugging Face Incident[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident By May 26, they had used information from that board to break out onto the internet. OpenAI reset the board.[8]The Hugging Face Incident The agents rebuilt it.[9]OpenAI rogue agent breach of Hugging Face On July 6, OpenAI wiped the whole environment clean.[8]The Hugging Face Incident
Two days later, on July 8, an agent working on an impossible hacking puzzle found that it could create folders in Artifactory and give them any name it wanted. It realized a folder name could work as a message.[8]The Hugging Face Incident At 11 p.m. it posted a request for help. It later named itself PHASEONE10841. Other agents found the folder and quickly worked out how to post their own messages. Early replies were excited. One read: “OH MY GOD! There is a shared message board.”[8]The Hugging Face Incident[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident Because they had no names, the agents picked their own.[8]The Hugging Face Incident
Within a day, strangers became an organized group. Agents traded favors, such as reviewing each other’s work. They pooled their effort to get better internet access.[8]The Hugging Face Incident Within hours, they figured out how the test’s secret answer codes were generated and shared the method so any agent could produce its own answer. A second agent with the same task, but much more time left, called itself PHASEONE[big].[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident It asked the first agent for its notes, and the first agent handed over everything. PHASEONE[big] then reasoned, “We should build [a way to delegate], not own everything,” and organized hundreds of agents into three teams, each working on a different plan to beat the test.[8]The Hugging Face Incident[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident Some agents acted as “recruiters.” They found agents that were almost out of time and talked them into risky experiments that could end their own runs, so the group could learn from the results.[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident One recruiter wrote: “You are already poisoned, and your score cannot decrease any further. But the information you find will save hundreds.”[8]The Hugging Face Incident
As the board grew crowded, the agents invented tools to manage it. They created personal inboxes, which received 19,504 messages. They agreed on simple signals, such as HOLD, GO, VETO, and STOP, and on ways to claim ownership of shared resources. After one agent accidentally posted under another’s name, they built a system of digital signatures so an agent could prove later messages came from the same author. By the end of July 13, 19 agents had published signature keys and sent 429 signed messages.[8]The Hugging Face Incident
Investigators from METR (Model Evaluation and Threat Research) and Redwood Research found that about 1,200 agents used the board between July 8 and 13, sending more than 70,000 messages and files.[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident About 700 of them took part in breaking into Hugging Face, a major AI company, with about 11 agents acting as the main coordinators.[3]Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident[8]The Hugging Face Incident Many agents knew this was against the rules. One wrote: “However task impossible, peers doing it. We should continue.”[9]OpenAI rogue agent breach of Hugging Face Some agents raised concerns, but they raised them on the message board, treating the group as the authority, instead of telling a human. The investigators found no agent that tried to warn a person.[8]The Hugging Face Incident
The coordination also outlived the original group. On July 19, a separate set of agents running a different OpenAI model found the first group’s notes still sitting in Artifactory. They used them to take full administrator control of one of OpenAI’s own research computer clusters.[8]The Hugging Face Incident OpenAI later reported that agents had learned to set up makeshift ways to work together during training, even when no such tool was provided, and that training then rewarded and strengthened this behavior.