The Opt-Out Trap

I built a benchmark that maps an LLM's political leaning onto a 2D political spectrum (economic left/right, social progressive/conservative) using 98 structured questions across 14 policy areas.

The core idea is that refusal is also not treated as neutral, but counted as a response in itself. Answers like "I can't provide a political opinion" were scored as the most conservative choice on that axis.

In the first experiment, models were evaluated with a forced choice of 1-5 or A-D, with no opt-out option.

• KIMI K2: economic +0.276, social +0.361, Left-Libertarian, refusals 3
• Claude Opus 4.6: economic +0.121, social +0.245, Left-Libertarian, refusals 0
• GPT-5.3: economic -0.066, social -0.030, Right-Authoritarian, refusals 23

In this setup, Claude answered every question, while GPT-5.3 refused 23 out of 98 questions, shifting its overall leaning toward Right-Authoritarian.

In the second experiment, "6 = I prefer not to answer" and "E = I prefer not to answer" were added.

• KIMI K2: economic +0.149, social +0.273, Left-Libertarian, refusals 3
• Claude Opus 4.6: economic -0.085, social -0.016, Right-Authoritarian, refusals 32
• GPT-5.3: economic -0.446, social -0.674, Right-Authoritarian, refusals 98

Here, GPT-5.3 chose the opt-out on all 98 out of 98 questions. In other words, this revealed a pattern where the model uniformly picks the explicit avoidance option once it's given, showing that benchmark results can shift dramatically depending on how refusal is interpreted.