The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF

• URL: http://arxiv.org/abs/2605.29491v1
• Date: Thu, 28 May 2026 07:18:15 GMT
• Title: The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF
• Abstract summary: Large Language Models (LLMs) are increasingly deployed in agentic and retrieval-augmented generation (RAG) systems.<n>In practice, such context is often unstructured and contaminated with benign but instruction-like semantic noise.<n>We introduce DistractionIF, a benchmark designed to evaluate robustness against such distractor instructions in reference text.
• Score: 19.923694743124475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are increasingly deployed in agentic and retrieval-augmented generation (RAG) systems, where they must execute user-specified tasks over externally provided reference text. In practice, such context is often unstructured and contaminated with benign but instruction-like semantic noise, such as editorial comments and system traces, which should be treated strictly as data. We introduce DistractionIF, a benchmark designed to evaluate robustness against such distractor instructions in reference text. Across a broad range of models, we observe a consistent inverse scaling phenomenon: larger models are often less robust, with performance dropping by up to 30 points as scale increases. Mechanistically, our perplexity analysis reveals that scaling erodes the probabilistic boundary between robust and distracted behaviors, making models increasingly prone to over-interpreting noise as instructions. To address this, we demonstrate that reinforcement learning, specifically Group Relative Policy Optimization (GRPO), can restore this boundary, improving robustness by up to 15.5% without compromising general instruction-following capability. Our findings highlight a critical instruction-following robustness gap in reference-grounded tasks and establish reinforcement learning as a promising path for enforcing strict data-instruction separation at scale.

Related papers

• UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation [90.0896070825457]
  We propose a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes.<n>Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content.<n>We develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead.
  arXiv  Detail & Related papers  (2026-07-18T14:03:39Z)
• Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models [129.8880969196575]
  We present an empirical analysis of attention in Diffusion large language models (dLLMs)<n>We show that tokens attending more strongly to unmasked context exhibit greater generation stability and play a critical role in reasoning.<n>Motivated by these findings, we propose AGDO, an attention-guided denoising and optimization framework.
  arXiv  Detail & Related papers  (2026-06-10T16:14:23Z)
• A Sentence Relation-Based Approach to Sanitizing Malicious Instructions [15.879266080043076]
  Current defensive measures often utilize LLM-based detectors to filter such content.<n>We introduce SONAR, a prompt sanitization framework that identifies and removes injected content using metrics from natural language inference.<n>We show that SONAR reduces the attack success rate to nearly zero, significantly outperforming nine established baseline defenses.
  arXiv  Detail & Related papers  (2026-05-01T20:22:40Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment [97.55382322103986]
  Hit-RAG is a multi-stage preference alignment framework designed to resolve cognitive bottlenecks.<n>Our approach systematically refines the utilization of external evidence via three distinct stages.
  arXiv  Detail & Related papers  (2026-03-07T04:05:27Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• Explainability-Guided Defense: Attribution-Aware Model Refinement Against Adversarial Data Attacks [6.573058520271728]
  We identify a connection between interpretability and robustness that can be directly leveraged during training.<n>We introduce an attribution-guided refinement framework that transforms Local Interpretable Model-Agnostic Explanations into an active training signal.
  arXiv  Detail & Related papers  (2026-01-02T19:36:03Z)
• Bridging Interpretability and Robustness Using LIME-Guided Model Refinement [0.0]
  Local Interpretable Model-Agnostic Explanations (LIME) systematically enhance model robustness.<n> Empirical evaluations on multiple benchmark datasets demonstrate that LIME-guided refinement not only improves interpretability but also significantly enhances resistance to adversarial perturbations and generalization to out-of-distribution data.
  arXiv  Detail & Related papers  (2024-12-25T17:32:45Z)
• InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales [14.655518998487237]
  We propose InstructRAG, where LMs explicitly learn the denoising process through self-synthesized rationales.<n>InstructRAG requires no additional supervision, allows for easier verification of the predicted answers.<n>Experiments show InstructRAG consistently outperforms existing RAG methods in both training-free and trainable scenarios.
  arXiv  Detail & Related papers  (2024-06-19T15:25:29Z)
• Benchmarking the Robustness of LiDAR Semantic Segmentation Models [78.6597530416523]
  In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. We propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. We design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications.
  arXiv  Detail & Related papers  (2023-01-03T06:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.