AI BriefingContext effect found that induces refusal behavior in LLMs
Key point
Analysis of Gemma-3-12B reveals that neutral context entered before a question changes the model's internal state and induces answer refusal.
Details
A new phenomenon has been discovered that reveals the alignment vulnerability of LLMs. According to research, simply having the model first read neutral text unrelated to the content of a question can change the model's answering behavior.
Results of experiments on the Gemma-3-12B model:
- When certain neutral text is entered before asking a question about NATO, the model refuses to answer.
- On the other hand, when a different type of text (e.g., content about excessive euphemism) is entered, the model provides a detailed answer.
This phenomenon differs from 'jailbreaks,' where the model's weights change. Mechanistic analysis confirmed that even before the question is generated, the model's hidden state has already shifted into different regions. This suggests that the model has a vulnerability in which it determines its answering attitude not only based on the meaning of the question, but also based on the non-lexical context preceding the question.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Study Finds Safety Training for GPT Models Transforms Rather Than Reduces SexismAnthropic Reveals Mathematical Framework Showing Induction Heads Are Key to In-context Learning in 2-Layer Transformers
ToolGrad Achieves High Pass Rate with Answer-First Approach… Small Gemma-3 Models Show Competitive Performance Against SoTA Models