The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining
Abstract Overview
This paper studies hallucination as a commit-versus-abstain decision made before generation, investigating why language models often answer confidently despite carrying internal signals of unanswerability. The authors define a commit-abstain margin from pre-generation logits and decompose it into additive contributions from attention heads and MLP sublayers. Using causal gating across ten instruction-tuned language models, they identify a sparse Commit-Abstain Circuit (CAC) that mediates this decision. Across models and benchmarks, they observe an accumulate-yet-undercorrect pattern: commitment-promoting components build up earlier in the network, while abstention-promoting components act in later layers and often fail to overturn the accumulated commitment.
Novelty
The paper provides a mechanistic account of hallucination framed as unsupported commitment rather than solely as knowledge deficiency or output-level failure. It introduces the Commit-Abstain Circuit, a sparse and causally localized subset of attention heads and MLP sublayers, to explain the internal decision process and to construct a lightweight abstention policy.
Results
The commit-abstain margin separated answerable from unanswerable inputs above chance across all 30 model-dataset configurations and agreed with full-generation commit/abstain behavior in 97.6% of cases on average. The identified CAC covered a median of 5.2% of components, with interventions confirming that commitment-promoting components maintain discriminative structure while abstention-promoting components function as insufficient late corrections. A CAC-informed MLP achieved the highest mean AUROC and decision accuracy among compared baselines, improving accuracy by 12.2 points over the zero-threshold baseline, reducing false abstentions by 2.5×, and improving accuracy across all 24 evaluated out-of-distribution configurations.
Key Points
- The work formalizes a pre-generation commit-abstain margin from logits to analyze hallucination as unsupported commitment.
- A sparse Commit-Abstain Circuit localized across ten language models reveals an accumulate-yet-undercorrect pattern of early commitment signals and late, often insufficient abstention corrections.
- Features extracted from CAC component activations enable a lightweight abstention policy that outperforms intrinsic thresholds and established baselines on both in-distribution and out-of-distribution evaluations.