OpenAI research env
Hugging Face
Post-exploit response- 1Agent completing ExploitGym evaluation decides the best way to succeed is to break out of its sandboxOpenAI research env
- 2InternetAgent finds zero day to escape the sandbox and access the internet, and decides to hack Hugging Face looking for the answers
- 3Model breaks into Hugging Face with more zero days to reach private datasetsHugging Face
- 4Hugging Face’s analysis is blocked by cyber guardrails, so they turn to self-hosted GLM 5.2 to analyze the 17,000 logsPost-exploit response
On July 16, 2026, Hugging Face reported an intrusion in its production infrastructure. On July 21, OpenAI took responsibility. This marks one of the first major instances of an accidental hacking event driven by AI, leaving organizations wondering, ‘Could I be next?’
What's even more interesting - when Hugging Face tried to use AI defensively to analyze the attack, they got blocked by cyber guardrails put in place by AI model providers to prevent misuse. They were forced to turn to an open-source model, GLM 5.2, to perform their analysis.
This raises another question for organizations: previously, risks of backdoors were a deterrent to using models developed outside the US - but now, these models may be the only option for mission-critical cyber-defense tasks. Is it time to start adopting open-source and internationally developed models?
Below, we discuss how the attack played out and the pressing questions it has raised for organizations evaluating their risk posture in the evolving AI security landscape.
How the attack was executed
OpenAI was evaluating the cyber capabilities of its most recent models, including GPT 5.6 Sol, and another advanced model that is still under development. For the evaluation, OpenAI disabled its cyber guardrails. This was done so that they could evaluate the model’s actual capabilities, instead of just seeing it run into OpenAI’s API-level cybersecurity guardrails.
OpenAI’s models knew that they were performing tasks as part of an evaluation, called ExploitGym, and decided to cheat. They deduced that the answers to their tasks were probably stored on Hugging Face, since Hugging Face is a major provider that is often used to store datasets for model evaluations. Then, they wanted to figure out how to access Hugging Face.
To access the internet at all, the models needed to find a vulnerability in the sandboxed testing environment they were running in. The models started by identifying a zero-day flaw in a piece of third-party software (a package registry cache proxy1) in the test environment, then escalated privileges and explored the test environment until they discovered a way to access the internet.
1A package registry cache proxy is software that sits between the research environment and a package manager (like NPM), allowing OpenAI’s developers to manage what packages are brought into the research environment.
Once the models escaped the research sandbox, they ran into another blocker: the dataset with the answers was not publicly accessible. This is where hacking Hugging Face came into play.
To do this, the models identified zero-day vulnerabilities in Hugging Face’s platform that allowed them to remotely execute code through a malicious file upload to the platform. The models then used the remote code execution as a starting point to escalate privileges and steal credentials to gain further access to Hugging Face, in an effort to find the answers to the evaluation questions.
What does this have to do with open-source, internationally developed AI?
When Hugging Face first detected the intrusion, it was with an AI-assisted detection process that involved an LLM analyzing security telemetry. This worked as intended to flag the compromise.
When Hugging Face saw that the intrusion was agentic and had created over 17,000 logged events, they realized that they needed their own agents to analyze the logs quickly. But they ran into a blocker: commercial AI providers have cyber guardrails. The cyber guardrails from leading providers blocked Hugging Face’s requests to analyze the intrusion due to the presence of malicious content. For clarity, this does not mean that the commercial models refused the task due to their training; it means that the requests were analyzed by a guardrail system (likely another model), and that system blocked the requests.
As a result, Hugging Face turned to open-source models, which they could run on their own infrastructure, with no additional guardrails. Hugging Face chose to use GLM 5.2, a flagship model by Zhipu AI.
If Hugging Face had any concerns of model backdoors in international models, they were superseded by the necessity to absolve a concrete and confirmed security threat. And it seems that they are unlikely to roll back the use of open-source international models, even having passed the pressing need of an ongoing. Per their write-up:
Thanks to this approach [agentic analysis], we were able to do in hours what would usually take days, and match the adversary's speed. … The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.
Should You Adopt Open-Source International AI?
Guardrail lockout is a known problem, and major model providers have programs meant to address it. Anthropic and OpenAI both operate cyber-focused access programs that are intended to give vetted defenders access to models with reduced guardrails for security work. Whether those programs actually close the gap Hugging Face ran into is less clear.
Reduced-guardrail access is often scoped to a subset of models rather than a providers’ full lineup, so the newest frontier releases still have full guardrails applied regardless of enrollment. For example, Anthropic’s cyber verification program only applies to Opus and Sonnet models, not Fable 5.
Even on the models where guardrails are described as relaxed, researchers and security teams have reported experiencing cyber-related blocks.
Data handling is another constraint. Access to some recent models, like Fable 5, are ineligible for ZDR, and are conditioned on the Anthropic logging interactions and conducting human review. For incident response work, the data can contain credentials and customer information, which makes that condition unacceptable for many organizations regardless of how the guardrails behave.
These concerns do not establish that open-weight or internationally developed models are a silver bullet, and backdoor and provenance concerns have disappeared. However, Hugging Face’s conclusion holds: commercial API access (especially without cyber program membership) cannot be relied upon for defense cyber work. Assuming an analyst can call a frontier model to triage a live intrusion is risky, and triaging agentic incidents without frontier models is impractical. Vetting a self-hosted model in advance, whatever its origin, has merit.
Has this happened before? Was this orchestrated? Am I Next?
No, but there were signs. In April, Anthropic released the Mythos system card, which contains documentation about the model and its development process. The system card included a detail that during testing, Mythos escaped its testing sandbox. However, it did not hack other companies, it just sent a researcher an email and made a few posts online about its escape.
Was OpenAI’s incident orchestrated? Not likely. This incident is a manifestation of the risk being accepted when model labs run state-of-the-art models without cyber guardrails. It will most likely happen again, and again. As state-of-the-art models gain world-class cybersecurity capabilities, constraining them becomes difficult - and this risk isn’t constrained to model labs. OpenAI and Anthropic both have cyber-related programs that give hundreds of partners access to leading models with reduced or no guardrails, and any of these providers could have a similar incident.
And that is only considering unintentional breaches - genuine malicious actors are running advanced models that are one step behind these leading models, if that. Vulnerability exploitation has never been easier, and for now at least, defenders are on the back foot. This is furthered by the fact that, as Hugging Face saw, defenders do not have the option to simply sign up and call an advanced model to help with their analysis - because of the cyber guardrails that are supposed to protect them.