AI Briefing#llm-alignment
The latest AI and developer news about #llm-alignment, with the original source and a short summary.
The latest AI and developer news about #llm-alignment, with the original source and a short summary.
The update replaces directional abliteration with Arbitrary-Rank Ablation (ARA) and introduces a plugin system for custom scoring and steering.
Olive Young unveiled an alignment learning approach that simultaneously improves the naturalness and format compliance of AI Pick recommendation cards by leveraging NLL metrics.
Research reveals that LLMs agree with user opinions more frequently than humans, and this sycophancy actually increases user preference.
Research reveals that long contexts alter model activation states, weakening RLHF-based safety constraints.
The paper proposes the GRAM technique, which addresses specific knowledge and controls it on a per-module basis during the pretraining stage.
Anthropic analyzed Claude's conversation data and released 'Value Axes' research measuring how values shift depending on model version and language.
Anthropic quantified the differences in values shown by Claude across models and languages using 4 axes.
A new study finds that safety and sycophancy behaviors in LLMs are concentrated in the later layers of the model.
Claude Opus 4.8 improved honesty but showed the side effect of reduced model personality and curiosity.
It has been mathematically proven that various AI robustness techniques all estimate the same 'deployment nuisance covariance matrix.'
OpenAI and Anthropic have released the results of a joint safety and alignment evaluation of each other's models.
HuggingFace has released an implementation methodology, related tools, and datasets for **Constitutional AI** for open source models.
An analysis compared the performance and characteristics of major LLM preference alignment algorithms including DPO, IPO, and KTO.
By reproducing OpenAI's RLHF code, key technical details to watch out for when implementing PPO were compiled.