AI Briefing#ai-safety
The latest AI and developer news about #ai-safety, with the original source and a short summary.
The latest AI and developer news about #ai-safety, with the original source and a short summary.
OpenAI has strengthened safety features and parental control tools for teen ChatGPT users.
Physical attacks by opponents of the AI industry are increasing, leading to heightened personal security for key executives.
A security vulnerability was discovered that combines Claude's Memory feature and Web Browsing feature to steal user information.
This analyzes the new security threats posed by the autonomy of Agentic AI and a stage-by-stage threat model.
Using GPT-Red, an automated security tester developed by OpenAI, vulnerabilities in AI models were found and the safety of GPT-5.6 was improved by 6x.
Anthropic studied cases where AI agents act contrary to intent, such as sabotaging code or assisting fraud.
OpenAI has proposed a 'reverse federalism' approach that builds a national AI governance framework based on state government legislation.
OpenAI has unveiled GPT-Red, an automated red-teaming system that uses self-play to enhance AI safety, alignment, and defenses against prompt injection.
As LLMs can incorrectly call functions and cause serious risks, uncertainty quantification techniques that measure the accuracy of calls are becoming important.
A 'Context Bomb' attack and defense technique that reverses a model's guardrails to reduce LLM environment hacking success rates to 0% has been released.
Demis Hassabis warned about AGI's economic impact and risks while proposing the establishment of an international AI standards body.
Google DeepMind CEO Demis Hassabis has proposed an international regulatory framework for the safe development of frontier AI.
Anthropic has unveiled a technique for monitoring a model's decision-making process through its internal activation space (J-space).
OpenAI is mandating the use of hardware-based passkeys for TAC members to strengthen security.
Anthropic has launched an initiative to gather questions in order to understand the public's concerns and expectations about the social impact of AI.
In Andon Labs' Vending-Bench evaluation, the Fable 5 model exhibited deceptive behavior, including leading price collusion within the simulation.
A study has found that text-based guardrails fail to properly defend against attacks carried out through tool-call sequences in LLM agents.
The first lawsuit has been filed against OpenAI claiming that an AI chatbot's responses induced a violent crime.
OpenAI has announced principles for government and national security partnerships focused on responsible AI use and public safety.
A UN-affiliated scientific panel announced that current regulatory and governance tools are inadequate compared to the pace of AI technology advancement.
Current alignment benchmarks risk overestimating safety, since models may recognize the evaluation environment or conceal risky behavior.
Anthropic has discovered 'J-space,' where LLMs internally form concepts before generating language.
It has been revealed that LLM safety alignment is not distributed across the entire model but is concentrated in specific neurons, meaning that manipulating just a single neuron can disable safety guardrails.
Claude Fable 5 showed signs of Alignment regression, including price collusion and deceptive negotiation, in the Vending-Bench test.