Can We Trust LLM Judges When They Agree?

In RAG system evaluation, agreement among multiple LLM judges on the same answer does not necessarily mean the result is reliable. This is because judges may share the same prompts or training lineage, leading them to repeat the same errors.

To address this issue, a paper presented at ICML proposes a method that uses the Ising model to analyze correlations between judges and adjust aggregated scores accordingly. This method simultaneously learns the individual reliability and similarity of judges, and can be trained solely on judge output data without human labels.

• Correlation adjustment: Redundant agreements are discounted to maintain the interpretability of prediction rules.
• Class-dependent model: Captures patterns where judges split into specific clusters on ambiguous items.

Across three binary classification tasks (relevance, toxicity, etc.), this method achieved 9-14% higher performance on standard metrics than historical accuracy-based weighted judges. It is also useful for determining whether additional judges provide independent evidence by leveraging agreement/disagreement patterns in existing evaluation logs.