Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

• URL: http://arxiv.org/abs/2607.11871v1
• Date: Mon, 13 Jul 2026 17:55:19 GMT
• Title: Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
• Abstract summary: We argue that the same biases admit a representation-level account in the judge's hidden state.<n>We report three findings, across seven judges, seven bias types, and nine benchmarks.
• Score: 28.271298519059034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/

Related papers

• How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? [53.58941416780391]
  We study where this susceptibility, spanning sycophancy and related cue-induced biases, lives inside the model.<n>We extract a per-bias direction from hidden states and triangulate it through three measures: probing, leave-one-dataset-out transfer, and causal intervention.
  arXiv  Detail & Related papers  (2026-07-20T16:10:06Z)
• Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering [0.0]
  Large Language Models are increasingly used as judges to evaluate code artifacts.<n>Current practice lacks a principled account of reliability and bias.<n>This paper studies LLM-as-a-Judge for code through a measurement-first lens.
  arXiv  Detail & Related papers  (2026-04-18T02:35:05Z)
• CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models [8.81733256907103]
  Large Language Models (LLMs) are increasingly deployed in high-stakes decision-making contexts.<n>We show that cognitive biases emerge systematically across all four families of cognitive biases: Judgment, Information Processing, Social, and Response.<n>We apply activation steering to modulate biased behavior, achieving 26--32% reduction in bias score.
  arXiv  Detail & Related papers  (2026-04-01T20:22:14Z)
• CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs [1.7697598440512727]
  We propose CatRAG Debiasing, a dual-pronged framework that integrates functor with Retrieval-Augmented Generation (RAG) guided structural debiasing.<n>On the Bias Benchmark for Question Answering (BBQ) across three open-source LLMs (Meta Llama-3, OpenAI GPT-OSS, and Google Gemma-3), CatRAG achieves accuracy by up to 40% over the corresponding base models and by more than 10% over prior debiasing methods.
  arXiv  Detail & Related papers  (2026-03-23T03:33:13Z)
• FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge [10.584937371987742]
  Existing LLM-as-a-Judge systems suffer from limited adaptivity to task- and domain-specific evaluation criteria.<n>We propose FairJudge, an adaptive, debiased, and consistent LLM-as-a-Judge.
  arXiv  Detail & Related papers  (2026-02-06T11:35:32Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Eliminating Position Bias of Language Models: A Mechanistic Approach [119.34143323054143]
  Position bias has proven to be a prevalent issue of modern language models (LMs)<n>Our mechanistic analysis attributes the position bias to two components employed in nearly all state-of-the-art LMs: causal attention and relative positional encodings.<n>By eliminating position bias, models achieve better performance and reliability in downstream tasks, including LM-as-a-judge, retrieval-augmented QA, molecule generation, and math reasoning.
  arXiv  Detail & Related papers  (2024-07-01T09:06:57Z)
• Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
  Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction. This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
  arXiv  Detail & Related papers  (2024-03-15T02:04:35Z)
• Large Language Models are not Fair Evaluators [60.27164804083752]
  We find that the quality ranking of candidate responses can be easily hacked by altering their order of appearance in the context. This manipulation allows us to skew the evaluation result, making one model appear considerably superior to the other. We propose a framework with three simple yet effective strategies to mitigate this issue.
  arXiv  Detail & Related papers  (2023-05-29T07:41:03Z)
• A Sandbox Tool to Bias(Stress)-Test Fairness Algorithms [19.86635585740634]
  We present the conceptual idea and a first implementation of a bias-injection sandbox tool to investigate fairness consequences of various biases. Unlike existing toolkits, ours provides a controlled environment to counterfactually inject biases in the ML pipeline. In particular, we can test whether a given remedy can alleviate the injected bias by comparing the predictions resulting after the intervention with true labels in the unbiased regime-that is, before any bias injection.
  arXiv  Detail & Related papers  (2022-04-21T16:12:19Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.