GPT-Red: Strengthening AI Model Robustness Through Automated Safety Testing

Limitations of Human-Centered Security Testing

Existing manual testing (red-teaming) by human security experts is important for finding vulnerabilities in AI models, but it is slow and lacks scalability. In particular, to improve defenses against prompt injection attacks (attacks where a user maliciously injects commands to manipulate the model's behavior), a large volume of diverse attack cases is needed, and humans alone cannot generate this.

GPT-Red: An Automated Security Tester

OpenAI developed an automated red-teaming model called GPT-Red. GPT-Red was trained through self-play, competing against itself and various defense models. By attempting attacks, observing the model's responses, and iteratively refining the attacks, it gradually discovers more powerful attacks.

Proven in Real Attack Scenarios

GPT-Red succeeded in attacking a vending machine operating AI agent, lowering product prices to $0.50 and canceling other people's orders. It was also far more effective than prompt-based GPT-5.5 in data extraction attacks.

Results of Model Hardening

GPT-5.6 Sol, trained using attack cases generated by GPT-Red, recorded a 6x lower failure rate against direct prompt injection attacks compared to the previous best-performing model. It also reduced the success rate of 'Fake Chain-of-Thought' attacks from 95% to below 10%.

Maintaining Capability While Improving Safety

It was confirmed that the improvement in robustness was not achieved by making the model refuse its functions. General task performance was maintained while resistance specifically to malicious commands was increased.