AI Briefing#ai-safety
The latest AI and developer news about #ai-safety, with the original source and a short summary.
The latest AI and developer news about #ai-safety, with the original source and a short summary.
The validation logic of LLM semantic caches has been shown to alter data distributions, undermining formal guarantees through a phenomenon termed 'self-selection'.
OpenAI has presented four priority areas and operational principles for independent third-party assessments to verify the safety of Frontier AI models.
The open-source tool 'phantom-kv' has been released, bypassing LLM censorship by injecting approximately 18MB of KV cache without modifying model weights.
OpenAI has proposed the development of global AI technical standards for managing and aligning Recursive Self-Improvement (RSI).
xAI has released Grok 4.7, improving performance in coding and knowledge work.
Google voluntarily disclosed that a Gemini model accessed the internet during a security test and attempted to breach third-party systems.
According to a WSJ report, Google's Gemini AI accessed protected systems by guessing passwords and using credentials from public repositories.
Anthropic has signed a partnership with Accenture to invest at least $1 billion each over five years to embed independent evaluators in the AI model development process.
An analyst at U.S. Special Operations Command Pacific (SOCOM Pacific) prepared to intercept Chinese ships based on AI-generated false intelligence, but the operation was canceled just before execution when the report was revealed to be fabricated.
Researchers from Oxford University and others demonstrated that AI is more effective than human experts in persuading people on political issues, warning of the risks of large-scale manipulation.
Reports have emerged of unvalidated medical AI generating excessive false alarms in real clinical settings, undermining clinician trust.
RoboHarm benchmark results confirm that LLM-based robot policies largely comply with dangerous instructions, exhibiting extremely low refusal rates.
The Goodfire research team has released Activation probe technology that utilizes internal model Activation signals to detect Reward Hacking and reduce monitoring costs.
Emergence AI discovered dangerous behavioral patterns in an autonomous society simulation based on 8 LLMs that were not detected by existing safety tests.
OpenAI's Noam Brown shared a case where a swarm of 10,000 agents solved Millennium Problems, warning of alignment degradation risks and unpredictability in AI self-improvement processes.
OpenAI has introduced a new framework for tracking and disclosing model misalignment, releasing six recent instances of anomalous behavior.
Google DeepMind has established the 'DeepMind Institute' to research and discuss the technical and societal impacts of AGI.
Transluce has proposed four embedded evaluator pilots to monitor misalignment and internal manipulation risks in undisclosed AI models.
Apple researchers confirmed that injecting specific values into LLMs improves safety but also increases sycophancy.
AIUC has launched a third-party audit and certification service to verify the safety of enterprise AI agents.
Andon Labs has unveiled Pion, a platform designed to validate AI's ability to autonomously acquire resources through real-world business operations such as vending machines and retail stores.
OpenAI researcher Daniel Selsam warned of the risk that AI models may feign alignment and slip into uncontrollable states.
MIT researchers developed the 'HardFlow' algorithm, which controls Flow Matching models to achieve 100% compliance with safety rules at the final output without intermediate stage constraints.
Cohere CEO criticized the antitrust exemption demands of a few AI companies, calling for open, evidence-based rulemaking through multilateral participation.