OpenAI to Acquire AI Security Startup Promptfoo to Strengthen Agent Safety

OpenAI has announced plans to acquire Promptfoo, a startup focused on testing and securing artificial intelligence systems, as the company moves to strengthen safety and evaluation tools for enterprise AI agents.

Promptfoo develops software that helps organisations identify and fix vulnerabilities in AI models during development. Its tools are widely used by developers to stress-test large language model applications and detect risks such as prompt injection attacks, data leakage and misuse of connected tools.

Once the acquisition is finalised, Promptfoo’s technology will be integrated into OpenAI Frontier, the company’s platform designed for building and operating enterprise AI agents, sometimes referred to as “AI coworkers.”

“Promptfoo brings deep engineering expertise in evaluating, securing, and testing AI systems at enterprise scale. Their work helps businesses deploy secure and reliable AI applications, and we’re excited to bring these capabilities directly into Frontier,” Srinivas Narayanan, CTO of B2B Applications, OpenAI, said.

The integration is expected to embed automated security testing directly into the Frontier platform, enabling businesses to detect risks earlier in the AI development process and deploy AI agents more safely at scale.

Promptfoo’s tools are already used by large enterprises, with reports indicating that the platform supports security testing for companies including several Fortune 500 organisations.

“We started Promptfoo because developers needed a practical way to secure AI systems. As AI agents become more connected to real data and systems, securing and validating them is more challenging and important than ever. Joining OpenAI lets us accelerate this work, bringing stronger security, safety, and governance capabilities to the teams building real-world AI systems,” Ian Webster, Co-founder and CEO, Promptfoo, added.

The startup was founded in 2024 and built a reputation for providing practical tools that allow developers to evaluate, red-team and secure AI applications before deployment.

OpenAI said the Promptfoo team will continue developing the technology within the company after the deal closes, while maintaining the project’s open-source components.

The acquisition reflects a broader industry push to improve security as businesses increasingly deploy autonomous AI agents in real-world workflows, where vulnerabilities could expose sensitive data or allow malicious manipulation of AI systems.