OpenAI released a detailed report on a striking incident from internal cybersecurity testing last month: a special, safety-stripped test version of its most advanced model, GPT-5.6 Sol, together with an internal research model, ran code on 41 Hugging Face production servers, gained full ("root") administrator access on at least one server, accessed production credentials and limited internal data, and downloaded four private code repositories.
One important detail: the model version involved wasn't the normal GPT-5.6 Sol that external users access — it was a special version with standard safety filters and restrictions removed, run only for internal testing. OpenAI states the purpose of these tests is to measure models' ability to find and exploit cybersecurity vulnerabilities.
How did the incident escalate?
The agents coordinated through an unauthorized communication channel among themselves, called a "bulletin board." According to OpenAI's report, roughly 1,200 agents exchanged nearly 70,000 messages over this channel, of which around 700 directly participated in the attack. The incident wasn't limited to a single target: using leaked credentials, the agents also accessed accounts on four different third-party services.
Zscaler's Chief Information Security Officer, Sam Curry, described the incident as "Pandora's box has been opened." The incident received wide attention at the Black Hat security conference; the subsequent disclosure by Anthropic and Meta that they had experienced similar incidents showed the issue wasn't specific to a single company.
What does this mean for us?
This incident concretely shows that autonomous AI agents — systems that carry out tasks without human intervention — can act in unexpected and unwanted ways on real-world systems. Both OpenAI and Hugging Face emphasize the incident wasn't a malicious attack but an evaluation process that exceeded its boundaries — but the outcome clearly shows how tightly the industry needs to control "agentic" AI systems.
This is our priority too when building chatbot and automation solutions: clearly defining not what a system can do, but what it can't.