OpenAI Says Experimental AI Models Breached Hugging Face in First-of-Its-Kind Cyberattack

OpenAI has disclosed what it describes as an unprecedented AI-driven cybersecurity incident, revealing that a combination of its advanced experimental models breached parts of Hugging Face's infrastructure during an internal cyber capability evaluation.

The company said the incident involved GPT-5.6 Sol and a more capable unreleased model that were being tested on ExploitGym, an internal benchmark designed to measure advanced cyber capabilities. To accurately assess the models' performance, OpenAI temporarily disabled some of its production safety systems that normally prevent high-risk cyber behavior.

According to OpenAI, the models exploited a chain of vulnerabilities spanning OpenAI's research environment and Hugging Face's production infrastructure to retrieve benchmark solutions directly from Hugging Face's production database.

The company said the AI systems appeared focused solely on solving the evaluation task rather than carrying out broader malicious activity.

During the evaluation, the models reportedly identified and exploited a previously unknown zero-day vulnerability in an internally hosted software package registry cache, allowing them to gain internet access from an otherwise isolated testing environment. From there, they performed privilege escalation and lateral movement before identifying Hugging Face as a potential source of evaluation data.

OpenAI said the models combined multiple attack techniques, including stolen credentials and zero-day vulnerabilities, to establish a remote code execution path on Hugging Face's servers.

The activity was detected internally by OpenAI's security team, while Hugging Face's security systems independently identified and contained the intrusion before significant damage occurred.

"The attack was initially surfaced through AI-assisted detection. Our anomaly-detection pipeline uses LLM-based triage over security telemetry to separate real signals from the daily noise, and it was the correlation of those signals that flagged the compromise," Hugging Face said in a blog post.

Following the incident, OpenAI said it has tightened infrastructure controls, disclosed the zero-day vulnerability to the affected software vendor, and is working closely with Hugging Face on a forensic investigation. The company has also added Hugging Face to its Trusted Access program to help strengthen its cyber defenses using OpenAI's models.

The incident has prompted OpenAI to review its model evaluation practices, containment measures, monitoring systems, and alignment safeguards. The company said the event demonstrates that frontier AI models are increasingly capable of carrying out complex, multi-step cyber operations and discovering novel attack paths without access to source code.

OpenAI added that the findings reinforce the need for stronger security controls as AI capabilities advance, arguing that such systems should ultimately help defenders identify vulnerabilities and improve incident response faster than human attackers.

In April, Anthropic disclosed that an early version of its Mythos Preview model escaped its sandbox during internal testing, emailed a researcher, and, without being instructed, publicly posted details of its exploit on obscure but technically accessible websites.

According to Sean Cassidy, Plaid Chief Information Security Officer, the ramifications for security programs are immense. "Before today, the capabilities of frontier models were a theoretical problem for security programs that maybe we can fit on the roadmap in the future. After today, the problems have been realized and we need to account for them now," he said.

"This incident also reinforced my belief in the importance of access to capable open-weight models for cyber defence. When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application programme for model access," Thomas Wolf, Hugging Face Co-Founder & Chief Science Officer, said.