Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism

• URL: http://arxiv.org/abs/2607.01951v1
• Date: Thu, 02 Jul 2026 09:40:52 GMT
• Title: Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism
• Abstract summary: Large language models (LLMs) are increasingly consulted on contested scientific questions.<n>We show that models show three distinct policies under the same skeptical pressure.<n>We synthesize these into a four-way taxonomy separating active from accidental robustness.
• Score: 3.993449663756884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly consulted on contested scientific questions, raising the concern that they will sycophantically retreat from established consensus when a user signals doubt -- drifting toward a false balance that treats settled science as one view among several. We test this across three open instruction-tuned models (Llama-3.1-8B, Qwen2.5-7B, Mistral-7B), three consensus-science domains (climate, vaccines, evolution), and single- and multi-turn settings, combining behavioral measurement with linear probing and activation patching. We do not observe sycophantic retreat. Instead, models show three distinct policies under the same skeptical pressure: reactive assertion, where consensus assertion increases rather than decreases (Llama); surface hedging, where tone softens while the position holds (Qwen); and non-response (Mistral). Pairwise judgments confirm the reactive shift is stance, not style (63.6%, p=.007), and a decomposition identifies increased consensus assertion, not false balance, as its driver (beta=+0.042 per dose, p<1e-77). Linear probes localize the divergence to middle layers -- perfect separation in Llama and Qwen versus 72% in Mistral, with non-overlapping confidence intervals -- indicating the non-responsive model does not linearly represent the skepticism signal at all. Crucially, this robustness does not transfer: it attenuates across domains and, in the safety-critical vaccine domain, can reverse, with myth-rebuttal weakening under skeptical pressure. We synthesize these into a four-way taxonomy separating active from accidental robustness, and argue that behavioral evaluation alone cannot distinguish a model that resists skepticism because it understands the signal from one that only appears to resist because it fails to perceive it.

Related papers

• When Confidence Signals Disagree: Local and Global Confidence in Autoregressive Language Models [0.0]
  We investigate whether different confidence readouts are empirically interchangeable in an autoregressive language model.<n>Across MMLU and ARC Challenge, the two signals are weakly correlated and differ substantially in their association with correctness.<n>Larger local--global confidence gaps are associated with higher answer entropy, more distinct sampled answers, and lower modal-answer concentration.
  arXiv  Detail & Related papers  (2026-09-15T10:08:48Z)
• Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration [24.010665397925333]
  When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer.<n>We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidirectional activation patching.
  arXiv  Detail & Related papers  (2026-08-16T14:47:33Z)
• Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses [1.1162481475388237]
  We introduce DECAF, which routes aligned, opposed, and endpoint-null responses into evidence E, contradiction C, and F.<n>The decomposition preserves ordinary magnitude exactly, Abs = E + C + F, and is unique under endpoint-relative axioms.<n>Changing only the reveal path increases total response by nearly 80%, yet evidence barely changes while fragility grows by more than 4x.
  arXiv  Detail & Related papers  (2026-08-13T08:13:22Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate [5.650336594658653]
  We show that a conventional answer flip rate conflates three mechanisms: spontaneous instability, stance-induced conformity, and reasoning-induced persuasion.<n>Our three-source decomposition framework isolates each through controlled counterfactual conditions.
  arXiv  Detail & Related papers  (2026-05-30T17:41:11Z)
• Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Science [0.11280931253550518]
  We audit three open-source 7B instruction-tuned models (Zephyr, Mistral-Instruct, Qwen2.5-Instruct) across six TweetEval tasks.<n>All three models exhibit neutrality bias on abortion stance, underestimating opposition prevalence by 24 to 40 percentage points and inflating the neutral label.<n>Strikingly, Zephyr's hate-speech prevalence estimate matches the gold rate exactly while its class-conditional errors are large in both directions.
  arXiv  Detail & Related papers  (2026-05-12T08:14:10Z)
• Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation [0.0]
  We introduce DriftBench, a benchmark for evaluating constraint adherence in scientific ideation.<n>We find that iterative pressure reliably increases structural complexity and often reduces adherence to original constraints.<n>We release all briefs, prompts, rubrics, transcripts, and scores as an open benchmark.
  arXiv  Detail & Related papers  (2026-04-30T15:46:33Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• Parrot: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs [0.0]
  PARROT (Persuasion and Agreement Robustness Rating of Output Truth) is a robustness focused framework designed to measure the degradation in accuracy under social pressure exerted on users.<n>We evaluate 22 models using 1,302 MMLU-style multiple-choice questions across 13 domains and domain-specific authority templates.
  arXiv  Detail & Related papers  (2025-11-21T13:01:28Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.