Ongoing Security Hardening of ChatGPT Atlas Against Prompt Injection Attacks

ChatGPT Atlas's Agent mode provides a powerful feature that views web pages, clicks, and performs keystrokes on behalf of the user within the browser. However, such agentic capabilities can become a high-value target for prompt injection attacks, making security extremely important.

Prompt injection is an attack that inserts malicious commands into content processed by an agent, causing the agent to act according to the attacker's intent rather than the user's. For example, if an agent reads a malicious email containing hidden commands while summarizing emails, it could result in damage such as sensitive documents being sent to the attacker without the user's permission.

To counter this, OpenAI utilizes automated Red Teaming based on Reinforcement Learning. Through this, OpenAI has built a response loop that internally discovers new attack strategies before real attacks occur and rapidly deploys security patches.

Recently, OpenAI deployed a security update that includes a new model trained with Adversarial Training and enhanced safeguards. OpenAI aims to stay ahead of attackers by leveraging white-box access to its models and large-scale compute resources.