Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning

• URL: http://arxiv.org/abs/2607.28478v1
• Date: Thu, 30 Jul 2026 16:30:08 GMT
• Title: Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning
• Abstract summary: We show that models become easily hijacked by useless explicit distractors, leading them to ignore the implicit physical or commonsense prerequisites of a task.<n>A critical open question is whether this failure reflects a genuine gap in commonsense knowledge or merely its suppression under misleading task framing.<n>Our findings relocate the bottleneck of commonsense reasoning failures from model competence to elicitation, and we release SaliTrap as a testbed for this blind spot.
• Score: 19.177595059532557
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) continue to advance in complex reasoning tasks, they have learned to heavily prioritize explicit conditions provided in the input. However, in everyday commonsense reasoning, this mechanism exposes a critical vulnerability which we term Salience Bias: models become easily hijacked by useless explicit distractors (e.g., numerical values), leading them to ignore the implicit physical or commonsense prerequisites of a task. A critical open question is whether this failure reflects a genuine gap in commonsense knowledge or merely its suppression under misleading task framing. To investigate this, we construct the SaliTrap Benchmark, a high-quality dataset across four trap dimensions. Evaluating 12 state-of-the-art LLMs, we find that all mainstream models suffer significantly from salience bias, with severity scaling with distractor density and detecting the trap often decoupled from actually avoiding it. Crucially, by re-eliciting the same models with the task framing stripped away, we show that this is overwhelmingly a failure of \textbf{knowledge suppression rather than knowledge absence}: a context-free knowledge probe alone recovers over 90\% of sycophantic-compliance failures, revealing that the requisite commonsense is intrinsically present but actively crowded out by salient distractors that lure the model into over-compliant, unnecessary computation. Building on this diagnosis, we further show that lightweight, inference-time prompting alone substantially closes the gap without any retraining. Our findings relocate the bottleneck of commonsense reasoning failures from model competence to elicitation, and we release SaliTrap as a testbed for this blind spot. The codes are available at https://github.com/Wuzheng02/SaliTrap.

Related papers

• Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning [109.33717747526583]
  Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks.<n>We introduce textbfCaRL (textbfCapability-textbfaligned textbfReinforcement textbfL) to align behavior with capability boundaries.<n>Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties.
  arXiv  Detail & Related papers  (2026-07-31T09:30:33Z)
• Unveiling the Visual Counting Bottleneck in Vision-Language Models [49.591496870141846]
  This work deconstructs visual counting into three cognitive stages: visual individuation, magnitude awareness, and symbolic mapping.<n>Using synthetic Go boards and linear probes, we demonstrate that visual backbones maintain robust, linearly separable representations of quantity well into the extrapolation regime.<n>We pinpoint the collapse to the symbolic mapping stage, where the model fails to project valid visual magnitudes onto symbolic tokens.
  arXiv  Detail & Related papers  (2026-05-28T16:20:29Z)
• Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination [25.643533134113607]
  We analyze the attention mechanisms via the logit lens, uncovering a distinct anomaly we term Vocabulary Hijacking.<n>We propose Hijacking-Aware Visual Attention Enhancement (HAVAE), a training-free intervention that selectively strengthens the focus of these identified heads on salient visual content.
  arXiv  Detail & Related papers  (2026-05-11T14:16:25Z)
• Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning [22.264933226031435]
  We present a comprehensive analysis using FaultyScience, a newly constructed large-scale, cross-disciplinary benchmark of faulty questions.<n>We show that the gap is pervasive and stems from token-level autoregression, which entangles task selection with content generation.<n>We propose DeIllusionLLM, a task-level autoregressive framework that explicitly models this decision.
  arXiv  Detail & Related papers  (2026-03-23T22:43:34Z)
• AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing [9.271196825503417]
  Large Language Models (LLMs) are increasingly adopted for vulnerability detection, yet their reasoning remains fundamentally unsound.<n>We present AEGIS, a novel multi-agent framework that shifts detection from ungrounded speculation to forensic verification over a closed factual substrate.<n>It reduces the false positive rate by up to 54.40% compared to leading baselines, at an average cost of $0.09 per sample without any task-specific training.
  arXiv  Detail & Related papers  (2026-03-21T04:12:04Z)
• Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning [16.274791437311602]
  We introduce ARMOR: Adaptive Round-based Multi-task mOdel for Robotic failure detection and reasoning.<n>We formulate detection and reasoning as a multi-task self-refinement process.<n>We show that ARMOR achieves state-of-the-art performance by improving over the previous approaches by up to 30% on failure detection rate.
  arXiv  Detail & Related papers  (2026-02-12T20:55:36Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Models [7.18947815679122]
  Internal State Probing and Chain-of-Thought Verification are used to detect hallucinations in large language models.<n>We develop a unified framework that bridges the gap between the two methods.<n>Our framework consistently and significantly outperforms strong baselines.
  arXiv  Detail & Related papers  (2025-10-13T15:31:21Z)
• Uncertainty-Aware Attention Heads: Efficient Unsupervised Uncertainty Quantification for LLMs [129.79394562739705]
  Large language models (LLMs) exhibit impressive fluency, but often produce critical errors known as "hallucinations"<n>We propose RAUQ (Recurrent Attention-based Uncertainty Quantification), an unsupervised approach that leverages intrinsic attention patterns in transformers to detect hallucinations efficiently.<n> Experiments across 4 LLMs and 12 question answering, summarization, and translation tasks demonstrate that RAUQ yields excellent results.
  arXiv  Detail & Related papers  (2025-05-26T14:28:37Z)
• Interpreting the Repeated Token Phenomenon in Large Language Models [31.1226642501095]
  Large Language Models (LLMs) often fail to accurately repeat a single word when prompted to, and instead output unrelated text.<n>We aim to explain the causes for this phenomenon and link it to the concept of attention sinks''<n>Our investigation identifies the neural circuit responsible for attention sinks and shows how long repetitions disrupt this circuit.
  arXiv  Detail & Related papers  (2025-03-11T21:40:58Z)
• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.