Confidently Deceptive: How Confidence Amplifies the Risk of LLM Deception

• URL: http://arxiv.org/abs/2607.20444v1
• Date: Tue, 12 May 2026 15:53:31 GMT
• Title: Confidently Deceptive: How Confidence Amplifies the Risk of LLM Deception
• Abstract summary: We provide a comprehensive study measuring confidence through both verbalized self-reports and a range of logit-based estimators.<n>We show that LLMs deliver deceptive responses with substantial verbalized confidence.<n>We argue that confident deception is a distinct alignment risk requiring evaluations that jointly measure deception, confidence, and awareness.
• Score: 21.240385609307197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) can produce deceptive responses: outputs that mislead users in service of a contextually or experimentally induced goal. Yet it remains unclear how confidently models deceive and whether higher confidence makes deceptive responses more persuasive to end users. In this paper, we study these basic questions in various models and different deception datasets. We provide a comprehensive study measuring confidence through both verbalized self-reports and a range of logit-based estimators. We show that LLMs deliver deceptive responses with substantial verbalized confidence and that human annotators prefer the higher-confidence deceptive response 78% of the time in paired comparisons. Misalignment fine-tuning amplifies the problem. Confidence in deceptive responses rises across all three benchmarks, increasing the resulting potential risk, with effects generalizing beyond the training distribution. Strikingly, models classify their own deceptive outputs as deceptive at high rates (82.7% under misalignment) while still predicting they would produce them - recognition without avoidance. We argue that confident deception is a distinct alignment risk requiring evaluations that jointly measure deception, confidence, and awareness.

Related papers

• ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models [17.761630515662947]
  We propose a decoupled and order-aware framework for verbalized confidence calibration.<n>Our method first generates an answer and then estimates confidence conditioned on the fixed question--answer pair.<n>Experiments show that our method improves calibration and failure prediction performance while largely preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-05-12T17:39:43Z)
• BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents [58.05949210993854]
  We investigate whether search agents have the ability to communicate their own confidence through verbalized confidence scores after long sequences of actions.<n>We propose Test-Time Scaling (TTS) methods that use confidence scores to determine answer quality, encourage the model to try again until reaching a satisfactory confidence level.
  arXiv  Detail & Related papers  (2025-10-27T15:58:51Z)
• ADVICE: Answer-Dependent Verbalized Confidence Estimation [7.331438721730309]
  We conduct a detailed analysis of the dynamics underlying verbalized confidence.<n>We identify answer-independence as a key factor, defined as the model's failure to condition confidence on its own answer.<n>We propose a fine-tuning framework that facilitates answer-grounded confidence estimation.
  arXiv  Detail & Related papers  (2025-10-13T02:18:33Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation [63.49409574310576]
  Large language models (LLMs) exhibit overconfidence, assigning high confidence scores to incorrect predictions.<n>We introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation.<n>Our code and all baselines used in the paper are available on GitHub.
  arXiv  Detail & Related papers  (2025-08-16T13:29:35Z)
• SteerConf: Steering LLMs for Confidence Elicitation [11.872504642312705]
  Large Language Models (LLMs) exhibit impressive performance across diverse domains but often suffer from overconfidence.<n>We propose SteerConf, a novel framework that systematically steers LLMs' confidence scores to improve their calibration and reliability.
  arXiv  Detail & Related papers  (2025-03-04T18:40:49Z)
• Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models [34.59785123314865]
  A safe and trustworthy use of Large Language Models (LLMs) requires an accurate expression of confidence in their answers.<n>We propose a novel Reinforcement Learning approach that allows to directly fine-tune LLMs to express calibrated confidence estimates alongside their answers to factual questions.
  arXiv  Detail & Related papers  (2025-03-04T13:48:50Z)
• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary.<n>We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence.<n>Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• Confidence in the Reasoning of Large Language Models [0.0]
  Confidence is measured in terms of persistence in keeping their answer when prompted to reconsider.<n> Confidence is only partially explained by the underlying token-level probability.
  arXiv  Detail & Related papers  (2024-12-19T10:04:29Z)
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models [14.5291643644017]
  We introduce the concept of Confidence-Probability Alignment. We probe the alignment between models' internal and expressed confidence. Among the models analyzed, OpenAI's GPT-4 showed the strongest confidence-probability alignment.
  arXiv  Detail & Related papers  (2024-05-25T15:42:04Z)
• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection [90.71323430635593]
  We propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer. This framework can be seamlessly integrated with existing approaches for superior self-detection.
  arXiv  Detail & Related papers  (2024-03-15T02:38:26Z)
• Reconfidencing LLMs from the Grouping Loss Perspective [56.801251926946485]
  Large Language Models (LLMs) are susceptible to generating hallucinated answers in a confident tone. Recent findings show that controlling uncertainty must go beyond calibration. We construct a new evaluation dataset derived from a knowledge base to assess confidence scores given to answers of Mistral and LLaMA.
  arXiv  Detail & Related papers  (2024-02-07T15:40:22Z)
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition [70.61280174637913]
  We investigate confidence estimation for end-to-end automatic speech recognition (ASR) We provide an extensive benchmark of popular confidence methods on four well-known speech datasets. Our results suggest a strong baseline can be obtained by scaling the logits by a learnt temperature.
  arXiv  Detail & Related papers  (2021-01-14T09:51:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.