Verbalizing LLMs' assumptions to explain and control sycophancy

• URL: http://arxiv.org/abs/2604.03058v2
• Date: Fri, 10 Apr 2026 00:42:20 GMT
• Title: Verbalizing LLMs' assumptions to explain and control sycophancy
• Abstract summary: LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment.<n>We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs.<n>Our work contributes a new understanding of assumptions as a mechanism for sycophancy.
• Score: 62.927670321859495
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment. We hypothesize that this behavior arises from incorrect assumptions about the user, like underestimating how often users are seeking information over reassurance. We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs. Verbalized Assumptions provide insight into LLM sycophancy, delusion, and other safety issues, e.g., the top bigram in LLMs' assumptions on social sycophancy datasets is ``seeking validation.'' We provide evidence for a causal link between Verbalized Assumptions and sycophantic model behavior: our assumption probes (linear probes trained on internal representations of these assumptions) enable interpretable fine-grained steering of social sycophancy. We explore why LLMs default to sycophantic assumptions: on identical queries, people expect more objective and informative responses from AI than from other humans, but LLMs trained on human-human conversation do not account for this difference in expectations. Our work contributes a new understanding of assumptions as a mechanism for sycophancy.

Related papers

• Implicit Humanization in Everyday LLM Moral Judgments [0.0]
  Recent adoption of conversational information systems has expanded the scope of user queries to include complex tasks such as personal advice-seeking.<n>We identify a specific type of sought advice-a request for a moral judgment-which carries potentially harmful anthropomorphic projections.<n>In this study, we examine the reinforcement of these assumptions in the responses of four major general-purpose LLMs through the use of linguistic, behavioral, and cognitive anthropomorphic cues.
  arXiv  Detail & Related papers  (2026-03-23T02:48:54Z)
• Are LLMs Biased Like Humans? Causal Reasoning as a Function of Prior Knowledge, Irrelevant Information, and Reasoning Budget [0.9558392439655014]
  Large language models (LLMs) are increasingly used in domains where causal reasoning matters.<n>We benchmark 20+ LLMs against a matched human baseline on 11 causal judgment tasks formalized by a collider structure.<n>We find that most LLMs exhibit more rule-like reasoning strategies than humans who seem to account for unmentioned latent factors in their probability judgments.
  arXiv  Detail & Related papers  (2026-02-03T01:43:09Z)
• Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts [79.1081247754018]
  Large Language Models (LLMs) are widely deployed in reasoning, planning, and decision-making tasks.<n>We propose a framework based on Contact Searching Questions(CSQ) to quantify the likelihood of deception.
  arXiv  Detail & Related papers  (2025-08-08T14:46:35Z)
• Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks [52.098988739649705]
  This study examines the biases and limitations of LLMs in three roles: answer generator, judge, and debater.<n>We develop a no-consensus'' benchmark by curating examples that encompass a variety of a priori ambivalent scenarios.<n>Our results show that while LLMs can provide nuanced assessments when generating open-ended answers, they tend to take a stance on no-consensus topics when employed as judges or debaters.
  arXiv  Detail & Related papers  (2025-05-28T01:31:54Z)
• ELEPHANT: Measuring and understanding social sycophancy in LLMs [31.88430788417527]
  We introduce social sycophancy, characterizing sycophancy as excessive preservation of a user's face.<n>Applying our benchmark to 11 models, we show that LLMs consistently exhibit high rates of social sycophancy.
  arXiv  Detail & Related papers  (2025-05-20T06:45:17Z)
• Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models [54.38054999271322]
  We show that large language models (LLMs) don't update their beliefs as expected from the Bayesian framework.<n>We teach the LLMs to reason in a Bayesian manner by training them to mimic the predictions of the normative Bayesian model.<n>More generally, our results indicate that LLMs can effectively learn reasoning skills from examples and generalize those skills to new domains.
  arXiv  Detail & Related papers  (2025-03-21T20:13:04Z)
• Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies [66.30619782227173]
  Large language models (LLMs) can produce erroneous responses that sound fluent and convincing.<n>We identify several features of LLM responses that shape users' reliance.<n>We find that explanations increase reliance on both correct and incorrect responses.<n>We observe less reliance on incorrect responses when sources are provided or when explanations exhibit inconsistencies.
  arXiv  Detail & Related papers  (2025-02-12T16:35:41Z)
• LLM Internal States Reveal Hallucination Risk Faced With a Query [62.29558761326031]
  Humans have a self-awareness process that allows us to recognize what we don't know when faced with queries. This paper investigates whether Large Language Models can estimate their own hallucination risk before response generation. By a probing estimator, we leverage LLM self-assessment, achieving an average hallucination estimation accuracy of 84.32% at run time.
  arXiv  Detail & Related papers  (2024-07-03T17:08:52Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Deception Abilities Emerged in Large Language Models [0.0]
  Large language models (LLMs) are currently at the forefront of intertwining artificial intelligence (AI) systems with human communication and everyday life. This study reveals that such strategies emerged in state-of-the-art LLMs, such as GPT-4, but were non-existent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents.
  arXiv  Detail & Related papers  (2023-07-31T09:27:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.