Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

• URL: http://arxiv.org/abs/2607.19345v1
• Date: Tue, 21 Jul 2026 17:59:21 GMT
• Title: Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
• Abstract summary: We show that models extensively copy text from the input into their reasoning traces rather than productively solving the problem.<n>By separating each prompt into task-relevant key evidence and irrelevant distractor context, we show that the root cause is insufficient grounding.<n>Motivated by this diagnosis, we propose GEAR, a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context.
• Score: 42.17614767170884
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical failure mode in this regime: \emph{repetitive copying}, where models extensively copy text from the input into their reasoning traces rather than productively solving the problem. We show that this behavior is pervasive across frontier long-context LLMs and intensifies with context length. By separating each prompt into task-relevant key evidence and irrelevant distractor context, we further show that the root cause is insufficient grounding: models copy from the prompt indiscriminately, and those that fail to focus on key evidence are far more likely to answer incorrectly. Motivated by this diagnosis, we propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context. To enable GEAR on natural-language data, we develop an automated pipeline that constructs evidence-annotated training data from arbitrary documents. We validate GEAR across multiple model scales and benchmarks, showing consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards, with larger gains at longer contexts, while also reducing repetitive copying and thinking length. Our findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.

Related papers

• SAGE: From Direct Answering to Evidence-Grounded Inference for Chinese Ancient Document Understanding [61.02068339726333]
  We propose SAGE, an evidence-grounded multi-agent framework for Chinese ancient document understanding.<n>SAGE coordinates specialized agents for task-aware planning, tool-mediated evidence acquisition, claim-level verification, and bounded replanning.<n> Experiments on the AncientDoc benchmark show that SAGE consistently outperforms matched direct-answering baselines.
  arXiv  Detail & Related papers  (2026-08-25T03:01:51Z)
• LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards [53.339700196282905]
  Reinforcement learning with verifiable rewards (RLVR) has shown promise for this task.<n>Existing methods are limited by low-confusability distractors and sparse, outcome-only reward signals.<n>textscLongTraceRL produces training contexts that are far more challenging than those built by random sampling or one-shot search.
  arXiv  Detail & Related papers  (2026-05-29T17:51:40Z)
• Pause or Fabricate? Training Language Models for Grounded Reasoning [50.104657152302956]
  We propose Grounded Reasoning via Interactive Reinforcement Learning (GRIL) for grounded reasoning under incomplete information.<n>GRIL decomposes the reasoning process into two stages: clarify and pause, which identifies whether the available information is sufficient, and grounded reasoning, which performs task solving once the necessary premises are established.<n>Experiments on GSM8K-Insufficient and MetaMATH-Insufficient show that GRIL significantly improves premise detection (up to 45%), leading to a 30% increase in task success while reducing average response length by over 20%.
  arXiv  Detail & Related papers  (2026-04-21T16:45:29Z)
• Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models [77.3748853516374]
  We propose a training-free, model-intrinsic grounding method for vision-language models.<n>We compute the entropy of the model's next-token distribution and backpropagate it to the visual token embeddings to obtain an entropy-gradient relevance map.<n>We then extract and rank multiple coherent regions to support multi-evidence queries, and introduce an iterative zoom-and-reground procedure with a spatial-entropy stopping rule.
  arXiv  Detail & Related papers  (2026-04-09T16:51:42Z)
• InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning [36.37924312794855]
  We argue that verbosity is not merely a length problem, but a symptom of poor intermediate reasoning quality.<n>We propose InfoDensity, a reward framework for RL training that combines an AUC-based reward and a monotonicity reward as a unified measure of reasoning quality.
  arXiv  Detail & Related papers  (2026-03-18T03:11:36Z)
• LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards [51.45138356629732]
  We introduce LongRLVR to augment the sparse answer reward with a dense and verifiable context reward.<n>This auxiliary signal directly incentivizes the model for selecting the correct grounding information.<n>LongRLVR consistently and significantly outperforms the standard RLVR across all models and benchmarks.
  arXiv  Detail & Related papers  (2026-03-02T18:07:53Z)
• Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewards [60.0970117192627]
  Reinforcement learning (RL) has emerged as a critical technique for enhancing LLM-based deep search agents.<n>Existing approaches primarily rely on binary outcome rewards, which fail to capture the comprehensiveness and factuality of agents' reasoning process.<n>We propose textbfCitation-aware RL Rewards (CaRR), a fine-grained reward framework for deep search agents.
  arXiv  Detail & Related papers  (2026-01-09T18:57:53Z)
• PoU: Proof-of-Use to Counter Tool-Call Hacking in DeepResearch Agents [24.502121097996294]
  Retrieval-augmented generation (RAG) agents extend large language models with autonomous information-seeking capabilities through external tools.<n>We identify a previously overlooked failure mode, Tool-Call Hacking, where agents inflate reward signals by issuing superficially correct tool calls.<n>We propose Proof-of-Use (PoU), an evidence-grounded RL framework that enforces verifiable causal links between retrieved evidence, reasoning traces, and final answers.
  arXiv  Detail & Related papers  (2025-10-13T02:45:37Z)
• Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation [19.551623461082617]
  Open-domain question answering answers a question based on evidence retrieved from a large corpus. This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training. We introduce a novel approach (DistDR) that iteratively improves over a weak retriever by alternately finding evidence from the up-to-date model and encouraging the model to learn the most likely evidence.
  arXiv  Detail & Related papers  (2021-10-10T20:01:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.