AI Briefing#red-teaming
The latest AI and developer news about #red-teaming, with the original source and a short summary.
The latest AI and developer news about #red-teaming, with the original source and a short summary.
Research by Mindgard found a vulnerability in which ChatGPT's image generation filters can be bypassed, resulting in the generation of inappropriate content.
OpenAI has introduced a new safety review method that predicts risky model behavior before launch by reproducing real conversational context through deployment simulation.
Anthropic warned that AI models are showing rapid capability improvements in cybersecurity and biology, signaling early indicators of national security risk.
Anthropic conducted red team testing with experts to evaluate AI frontier threats directly tied to national security, such as biological risks.
This analyzes the various Red Teaming methodologies used to secure the safety of AI systems, along with their respective pros, cons, and challenges.
An autonomous hacking agent that performs professional red team attacks using 16 specialized agents and LangGraph.
Anthropic appears to have started red teaming Jupiter V1 ahead of its May 6 developer conference.
Interaction vulnerabilities were revealed in a network of over 100 AI agents.
OpenAI has launched a bug bounty to find jailbreaks that bypass GPT-5.5's bio safety measures.
OpenAI is offering $25,000 for a universal jailbreak that breaks through GPT-5.5's five bio safety questions.
The US government is testing Claude Mythos's security capabilities despite the ban on Anthropic.
In 50-turn conversations, the system prompt gets diluted and agents miss safety warnings.
OpenAI is acquiring the security platform Promptfoo to strengthen the security and evaluation capabilities of AI agents.
Hallucination, bias, jailbreaking, and sycophancy in generative AI are threatening safety and trust.
OpenAI continuously strengthens ChatGPT Atlas's prompt injection defenses through automated red teaming techniques.
As AI's cybersecurity capabilities improve, OpenAI is strengthening a multi-layered security strategy to support defenders and prevent misuse.
Stability AI has released its annual integrity transparency report to demonstrate the safety and transparency of its model development and deployment process.
Anthropic worked with US and UK AI security institutions to test vulnerabilities in Claude models and strengthen safeguards.
OpenAI is collaborating with US CAISI and UK AISI to examine security vulnerabilities in AI agents and strengthen its systems.
OpenAI is running a Bio Bug Bounty to strengthen GPT-5's defenses against biological risks.
Anthropic lowered the universal jailbreak success rate to 4.4% with Constitutional Classifiers.
Anthropic has developed a multi-pronged response to prevent AI misuse during the 2024 global election season, including strengthened policies, model testing, and provision of accurate information.
Anthropic implements robust safety measures based on 'Safety by Design' principles to prevent child sexual exploitation and sexual harm through generative AI.
Anthropic implemented safety measures for Claude during the 2024 election period, and Clio analysis showed election-related usage was very low.