Untangling the Mechanisms of Misleading Context in Medical Question Answering

• URL: http://arxiv.org/abs/2609.02754v1
• Date: Wed, 02 Sep 2026 15:55:56 GMT
• Title: Untangling the Mechanisms of Misleading Context in Medical Question Answering
• Abstract summary: Large language models now answer medical questions with expert-level performance.<n>We examine the model's susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision.
• Score: 3.6576251307216783
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models now answer medical questions with expert-level performance. However, the context these systems act on can be misleading, and misleading context can corrupt a model's medical judgment. To understand how misleading context corrupts this judgment, we examine the model's susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision. On the medical reasoning subset of MedMisBench, a clinician-reviewed question-answering benchmark of 8,627 questions, we inject two types of misleading context cues, fabricated evidence and a bare assertion. We test three reasoning models, two that expose their full reasoning trace and one frontier model that exposes only its response. All three are more susceptible to the assertion than to the fabricated evidence, adopting the asserted answer 10 to 27 points more often. The misleading cues are disclosed in 81 to 98% of traces but only 7 to 90% of responses, and the assertion is disclosed less often than evidence based cues. Resampling from reasoning traces without disclosure shows the two cues corrupt reasoning differently, evidence entering early and accumulating while the assertion redirects the conclusion near its end. An LLM monitor catches 78% of corrupted decisions at 5% false positives when reading an open model's trace with guidance, against at most 32% from any response. The misleading context that models are most susceptible to is disclosed least, and was caught reliably only from an open reasoning trace, which frontier providers withhold.

Related papers

• Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents [56.93800079330285]
  Existing multimodal benchmarks largely evaluate final-answer accuracy.<n>We introduce Sci-MMR, a benchmark for multi-step evidence-grounded scientific reasoning.<n>We find that answer accuracy consistently exceeds complete-evidence recovery rate by more than 20%.
  arXiv  Detail & Related papers  (2026-09-10T08:41:07Z)
• FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect [69.8983512616053]
  Large language models often change their responses to subtle rephrasings that align with an implied stance by users.<n>This can leave users with advice tainted by how they happened to phrase a question rather than by the underlying facts.<n>We introduce FramingQA, a benchmark that measures the model sensitivity to question framing across law, medicine, finance, and robotic simulations.
  arXiv  Detail & Related papers  (2026-09-07T12:56:44Z)
• Why LLMs Give In: Conversational Factors and Reasoning Behind Medical Sycophancy [8.386215461053723]
  A language model that abandons a correct medical answer under user pushback is more dangerous than one that was simply wrong.<n>We study medical sycophancy in language models with a fully crossed factorial design over four conversational factors.
  arXiv  Detail & Related papers  (2026-08-02T05:49:25Z)
• Reasoning Traces Shape Outputs but Models Won't Say So [11.928107300132062]
  We introduce Thought Injection, a method that injects synthetic reasoning snippets into a model's think> trace.<n>We find that injected hints reliably alter outputs, confirming that reasoning traces causally shape model behavior.<n>When asked to explain their changed answers, models overwhelmingly refuse to disclose the influence.
  arXiv  Detail & Related papers  (2026-03-21T03:31:36Z)
• Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning [0.0]
  We conduct a systematic evaluation of faithfulness in medical reasoning among three widely used closed-source large language models.<n>We find that CoT reasoning steps often do not causally drive predictions, and models readily incorporate external hints without acknowledgment.
  arXiv  Detail & Related papers  (2026-03-14T15:30:53Z)
• The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence [49.94160400740222]
  We introduce MisBelief, a framework that generates misleading evidence via collaborative, multi-round interactions.<n>Using MisBelief, we generate 4,800 instances across three difficulty levels to evaluate 7 representative LLMs.<n>Results indicate that while models are robust to direct misinformation, they are highly sensitive to this refined evidence.<n>We propose Deceptive Intent Shielding (DIS), a governance mechanism that provides an early warning signal by inferring the deceptive intent behind evidence.
  arXiv  Detail & Related papers  (2026-01-09T02:28:00Z)
• CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate [3.2852123901391077]
  We develop a dual-agent debate framework for causal inference.<n>Agents attempt to persuade each other, challenging each other's logic.<n>We show that strong models can still benefit greatly from debate with weaker agents.
  arXiv  Detail & Related papers  (2025-11-28T03:19:35Z)
• Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations [60.27156500679296]
  We study the role of Chain-of-Thought (CoT) explanations in moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones.<n>Our findings reveal two key effects: (1) users often trust with outcome agreement, sustaining reliance even when reasoning is flawed.<n>These results highlight how CoT explanations can simultaneously clarify and mislead, underscoring the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust.
  arXiv  Detail & Related papers  (2025-11-15T02:38:49Z)
• Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios [49.53589774730807]
  Multimodal large language models (MLLMs) have recently achieved state-of-the-art performance on tasks ranging from visual question answering to video understanding.<n>We reveal a response uncertainty phenomenon: twelve state-of-the-art open-source MLLMs overturn a previously correct answer in 65% of cases after receiving a single deceptive cue.
  arXiv  Detail & Related papers  (2024-11-05T01:11:28Z)
• MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries? [70.77691645678804]
  Humans are prone to cognitive distortions -- biased thinking patterns that lead to exaggerated responses to specific stimuli. This paper demonstrates that advanced Multimodal Large Language Models (MLLMs) exhibit similar tendencies. We identify three types of stimuli that trigger the oversensitivity of existing MLLMs: Exaggerated Risk, Negated Harm, and Counterintuitive.
  arXiv  Detail & Related papers  (2024-06-22T23:26:07Z)
• Missci: Reconstructing Fallacies in Misrepresented Science [84.32990746227385]
  Health-related misinformation on social networks can lead to poor decision-making and real-world dangers. Missci is a novel argumentation theoretical model for fallacious reasoning. We present Missci as a dataset to test the critical reasoning abilities of large language models.
  arXiv  Detail & Related papers  (2024-06-05T12:11:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.