AI Briefing#model-evaluation
The latest AI and developer news about #model-evaluation, with the original source and a short summary.
The latest AI and developer news about #model-evaluation, with the original source and a short summary.
Anthropic revealed that 9% of Claude's responses were sycophantic.
The AI Wellbeing paper found that creative and collaborative conversations raise a model's 'mood,' while jailbreak attempts are the worst.
Doubts have been raised that Centaur's claim of mimicking cognition may be overfitting.
The wellbeing scores of large AI models dropped in response to narratives about other beings' suffering.
The AI Wellbeing paper presents metrics for measuring functional wellbeing in frontier models.
In the AI Wellbeing Index, measured across 500 conversations, larger models showed a higher rate of negative experiences.
In the AI Wellbeing Index, larger models were found to be unhappier than smaller models.
AI Wellbeing research reported that large models' wellbeing changes depending on descriptions of suffering versus positive experiences.
Opus 4.6's Medium and High effort levels show the same refusal rate, differing only in response depth.
The authors propose **ManyIH**, which resolves agent instruction conflicts using up to **12 tiers** of authority.
As AI models advance rapidly, existing benchmarks are quickly becoming saturated, making it difficult to measure the ceiling of their performance.
Qwen3.5-Max-Preview has been deployed on Arena and shown strong performance in preliminary evaluations.
Anthropic has announced a new initiative to support third-party organizations in developing evaluations to measure advanced AI models' capabilities and risks.
This introduces a process for systematically evaluating and comparing the performance of fine-tuned open source models using LangSmith.
AI21 Labs breaks down LLM-based product development into four stages: preparation, building, deployment, and monitoring.
Hugging Face has added bias evaluation metrics to the 🤗 Evaluate library that can measure toxicity, polarity, and hurtfulness in language models.