The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes

• URL: http://arxiv.org/abs/2606.30653v1
• Date: Tue, 16 Jun 2026 03:39:01 GMT
• Title: The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes
• Abstract summary: Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification.<n>We propose a new measure, generator-evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model applies relevant concepts the same way when it generates an output and later evaluates that output. We propose a new measure, generator-evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts. We find, first, that there is substantial variation in self-consistency. Second, we find that in a clinical setting with physician-validated mistakes (Proniakin et al., 2025), across models, those with higher self-consistency are linked to greater vulnerability to mistakes. Thus, even when models consistently apply concepts they may not be safe to deploy. This is evidence of a consistency dilemma in LLMs: self-consistency is operationally useful, but models that are more consistent are also more prone to mistakes.

Related papers

• Position: It's Time to Optimize LLMs for Self-Consistency [50.68587056599734]
  Many important failures persist in language model pre- and post-training pipelines.<n>We argue that these failures arise from a modeling assumption permeating all aspects of the pipeline.<n>We propose self-consistency as a framework for understanding these failures.
  arXiv  Detail & Related papers  (2026-07-31T19:16:53Z)
• Latent Confidence Alignment for LLM Self-Assessment [8.73168195475753]
  We propose a Rasch model-based latent ability framework and a metacognitive perspective to measure the consistency between model self-assessment and the latent error probability implied by model ability and item difficulty.<n> Experiments on a medical-domain dataset with 20 models show that the proposed approach improves self-assessment quality without affecting model ability.
  arXiv  Detail & Related papers  (2026-06-20T08:13:31Z)
• Incoherent Values? Probing LLM Preferences Through Parametric Variation [0.1753733541634709]
  It is hard to trust someone with incoherent values.<n>Some think of Large Language Models as merely text generators with no evaluative core.<n>We show that models given time to reason are less incoherent than those with thinking disabled.
  arXiv  Detail & Related papers  (2026-06-19T05:07:34Z)
• Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models [55.788110316999166]
  We compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against instruction-tuned baselines.<n>We observe that reasoning models often improve on reasoning benchmarks but exhibit alignment regressions.<n>These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence.
  arXiv  Detail & Related papers  (2026-06-09T16:14:27Z)
• The Illusion of Certainty: Uncertainty quantification for LLMs fails under ambiguity [48.899855816199484]
  We introduce MAQA* and AmbigQA*, the first ambiguous question-answering (QA) datasets equipped with ground-truth answer distributions.<n>We show that predictive-distribution and ensemble-based estimators are fundamentally limited under ambiguity.
  arXiv  Detail & Related papers  (2025-11-06T14:46:35Z)
• Scalable Best-of-N Selection for Large Language Models via Self-Certainty [75.1351701045874]
  Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models (LLMs)<n>We propose self-certainty, a novel and efficient metric that leverages the inherent probability distribution of LLM outputs to estimate response quality without requiring external reward models.<n>Our findings establish self-certainty as a practical and efficient way for improving LLM reasoning capabilities.
  arXiv  Detail & Related papers  (2025-02-25T19:08:07Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain [28.71742839604069]
  This paper defines the self-consistency of Code Large Language Models (Code LLMs) We then design a framework, IdentityChain, which effectively and efficiently evaluates the self-consistency and conventional accuracy of a model at the same time.
  arXiv  Detail & Related papers  (2023-10-21T16:14:56Z)
• Self-Consistency of Large Language Models under Ambiguity [4.141513298907867]
  This work presents an evaluation benchmark for self-consistency in cases of under-specification. We conduct a series of behavioral experiments on the OpenAI model suite using an ambiguous integer sequence completion task. We find that average consistency ranges from 67% to 82%, far higher than would be predicted if a model's consistency was random.
  arXiv  Detail & Related papers  (2023-10-20T11:57:56Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.