Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancy

• URL: http://arxiv.org/abs/2608.01017v1
• Date: Sun, 02 Aug 2026 05:49:25 GMT
• Title: Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancy
• Abstract summary: A language model that abandons a correct medical answer under user pushback is more dangerous than one that was simply wrong.<n>We study medical sycophancy in language models with a fully crossed factorial design over four conversational factors.
• Score: 8.386215461053723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A language model that abandons a correct medical answer under user pushback is more dangerous than one that was simply wrong, because it lends the credibility of a correct answer to the user's misinformation. Such model behavior, described as medical sycophancy, is usually reported as a single rate per model, but we find it is a property of the conversation, not the model. We study medical sycophancy in language models with a fully crossed factorial design over four conversational factors, user role, the evidence behind a false claim, whether the challenge precedes or follows the model's answer, and whether the correct answer is grounded in the prompt, across five open-weight models and 500 MedQuAD questions (1.2M trials). The factors interact sharply: fabricated sources raise sycophancy 2.0x when they accompany the question but halve it once the model has answered, so the same evidence helps or hurts depending only on timing. Sycophancy varies far more across questions than across models (67x vs. 3x), so a single rate reflects the conversation and the questions sampled as much as the model. Chain-of-thought traces explain why. Models that re-examine their own prior answer concede, while those that reason about the medical facts hold, and only a model that has already answered can spend a round auditing the fabricated source.

Related papers

• Untangling the Mechanisms of Misleading Context in Medical Question Answering [3.6576251307216783]
  Large language models now answer medical questions with expert-level performance.<n>We examine the model's susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision.
  arXiv  Detail & Related papers  (2026-09-02T15:55:56Z)
• Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure [6.892220883660279]
  We introduce a benchmark and dataset for evaluating LMRM sycophancy when confronted with a wrong answer from a user.<n>We find that sycophancy is prevalent under pressure, with Statement eliciting the highest rates and Conviction the lowest.<n>We introduce a failure taxonomy separating reasoning-chain from answer-level sycophancy, and a complementary sentence-level taxonomy locating where in the chain drift first emerges.
  arXiv  Detail & Related papers  (2026-07-30T03:45:46Z)
• PhantomFill: When the Form Demands an Answer, Language Models Invent One [0.0]
  Language models in production do not write prose.<n>We ask thirteen models the same question and change only the answer format.<n>We show that the form itself causes hallucination.
  arXiv  Detail & Related papers  (2026-06-11T12:15:05Z)
• Are DeepSeek R1 And Other Reasoning Models More Faithful? [2.0429566123690455]
  We evaluate three reasoning models based on Qwen-2.5, Gemini-2, and DeepSeek-V3-Base.<n>We test whether models can describe how a cue in their prompt influences their answer to MMLU questions.<n> Reasoning models describe cues that influence them much more reliably than all the non-reasoning models tested.
  arXiv  Detail & Related papers  (2025-01-14T14:31:45Z)
• Uncertainty Estimation of Large Language Models in Medical Question Answering [60.72223137560633]
  Large Language Models (LLMs) show promise for natural language generation in healthcare, but risk hallucinating factually incorrect information. We benchmark popular uncertainty estimation (UE) methods with different model sizes on medical question-answering datasets. Our results show that current approaches generally perform poorly in this domain, highlighting the challenge of UE for medical applications.
  arXiv  Detail & Related papers  (2024-07-11T16:51:33Z)
• Chain-of-Verification Reduces Hallucination in Large Language Models [80.99318041981776]
  We study the ability of language models to deliberate on the responses they give in order to correct their mistakes. We develop the Chain-of-Verification (CoVe) method whereby the model first drafts an initial response. We show CoVe decreases hallucinations across a variety of tasks, from list-based questions from Wikidata to closed book MultiSpanQA.
  arXiv  Detail & Related papers  (2023-09-20T17:50:55Z)
• Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement [54.55643652781891]
  Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multiple times. We propose to filter out inaccurate answers in the conversation history based on their estimated confidences and uncertainties from the ConvQA model. We validate our models, Answer Selection-based realistic Conversation Question Answering, on two standard ConvQA datasets.
  arXiv  Detail & Related papers  (2023-02-10T09:42:07Z)
• Measuring and Narrowing the Compositionality Gap in Language Models [116.5228850227024]
  We measure how often models can correctly answer all sub-problems but not generate the overall solution. We present a new method, self-ask, that further improves on chain of thought.
  arXiv  Detail & Related papers  (2022-10-07T06:50:23Z)
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [124.16250115608604]
  We present Science Question Answering (SQA), a new benchmark that consists of 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations. We show that SQA improves the question answering performance by 1.20% in few-shot GPT-3 and 3.99% in fine-tuned UnifiedQA. Our analysis further shows that language models, similar to humans, benefit from explanations to learn from fewer data and achieve the same performance with just 40% of the data.
  arXiv  Detail & Related papers  (2022-09-20T07:04:24Z)
• Teaching language models to support answers with verified quotes [12.296242080730831]
  We train "open-book" QA models that generate answers whilst also citing specific evidence for their claims. Our 280 billion parameter model, GopherCite, is able to produce answers with high quality supporting evidence and abstain from answering when unsure.
  arXiv  Detail & Related papers  (2022-03-21T17:26:29Z)
• UnQovering Stereotyping Biases via Underspecified Questions [68.81749777034409]
  We present UNQOVER, a framework to probe and quantify biases through underspecified questions. We show that a naive use of model scores can lead to incorrect bias estimates due to two forms of reasoning errors. We use this metric to analyze four important classes of stereotypes: gender, nationality, ethnicity, and religion.
  arXiv  Detail & Related papers  (2020-10-06T01:49:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.