Context Recycling for Long-Horizon LLM Inference

• URL: http://arxiv.org/abs/2606.26105v1
• Date: Fri, 01 May 2026 04:45:08 GMT
• Title: Context Recycling for Long-Horizon LLM Inference
• Abstract summary: Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons.<n>We introduce ContextForge, a system for context recycling that maintains task-relevant information across turns.<n>System enables efficient reuse of prior computation without relying on full context replay.
• Score: 0.3655021726150367
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons due to context window limitations and inefficient token usage. We introduce ContextForge, a system for context recycling that maintains task-relevant information across turns by combining structured query generation, external memory retrieval, and controlled synthesis. The system enables efficient reuse of prior computation without relying on full context replay, reducing token overhead while preserving answer quality. We evaluate ContextForge using a 15-turn conversational benchmark that tests multi-turn reasoning, back-references, and domain shifts across structured healthcare queries. Compared to a baseline agent using identical underlying models, ContextForge demonstrates improved consistency and reduced token consumption, while maintaining comparable response accuracy. These results suggest that context recycling provides a practical approach for extending LLM capabilities in long-horizon tasks without requiring larger context windows or model retraining. Code and evaluation artifacts are available at https://github.com/Betanu701/ContextForge.

Related papers

• Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size [0.6155604731137828]
  Large language models (LLMs) process long contexts, including long documents and lengthy conversations.<n>LLMs face token-level memory usage that increases proportionally to input length.<n>This paper proposes a long-context recall method that maintains near-constant GPU memory usage as context length increases.
  arXiv  Detail & Related papers  (2026-09-11T10:31:46Z)
• RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models [54.53236295237077]
  We introduce RepoReasoner, a benchmark for evaluating repository-level code reasoning.<n> Output Prediction measures fine-grained, stateful execution reasoning across files.<n>Call Chain Prediction evaluates high-level architectural dependency understanding under noisy context.
  arXiv  Detail & Related papers  (2026-07-28T17:12:41Z)
• ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning [78.94047086588078]
  We propose Recursive Evidence as LLM Harness for Long-Context Reasoning.<n>It is a training-free inference method for improving long-context reasoning.<n> Experiments on eight long-context datasets with 128K context length show it consistently improves evidence utilization.
  arXiv  Detail & Related papers  (2026-07-02T17:59:26Z)
• REFRAG: Rethinking RAG based Decoding [67.4862300145604]
  REFRAG is an efficient decoding framework that compresses, senses, and expands to improve latency in RAG applications.<n>We provide rigorous validation of REFRAG across diverse long-context tasks, including RAG, multi-turn conversations, and long document summarization.
  arXiv  Detail & Related papers  (2025-09-01T03:31:44Z)
• RepoScope: Leveraging Call Chain-Aware Multi-View Context for Repository-Level Code Generation [16.544483144957407]
  RepoScope is a call chain-aware multi-view context for repository-level code generation.<n>We present a novel call chain prediction method that utilizes the repository's structural semantics to improve the identification of callees in the target function.<n>RepoScope outperforms state-of-the-art methods, achieving up to a 36.35% relative improvement in pass@1 scores.
  arXiv  Detail & Related papers  (2025-07-20T02:35:36Z)
• ELITE: Embedding-Less retrieval with Iterative Text Exploration [5.8851517822935335]
  Large Language Models (LLMs) have achieved impressive progress in natural language processing.<n>Their limited ability to retain long-term context constrains performance on document-level or multi-turn tasks.
  arXiv  Detail & Related papers  (2025-05-17T08:48:43Z)
• Context-DPO: Aligning Language Models for Context-Faithfulness [80.62221491884353]
  We propose the first alignment method specifically designed to enhance large language models' context-faithfulness.<n>By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization.<n>Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular open-source models.
  arXiv  Detail & Related papers  (2024-12-18T04:08:18Z)
• DoubleDipper: Improving Long-Context LLMs via Context Recycling [44.24067814871803]
  We propose DoubleDipper, a novel In-Context-Learning method that automatically generates few-shot examples for long context QA tasks.<n>We apply our method on multiple Large Language Models and obtain substantial improvements.<n>Surprisingly, despite introducing only single-hop ICL examples, LLMs successfully generalize to multi-hop long-context QA.
  arXiv  Detail & Related papers  (2024-06-19T15:28:29Z)
• On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing [82.96523584351314]
  We decouple the task of context retrieval from the other components of the repository-level code editing pipelines. We conclude that while the reasoning helps to improve the precision of the gathered context, it still lacks the ability to identify its sufficiency.
  arXiv  Detail & Related papers  (2024-06-06T19:44:17Z)
• LLoCO: Learning Long Contexts Offline [63.3458260335454]
  We propose LLoCO, a novel approach to processing long contexts. LLoCO learns contexts offline through context compression and in-domain parameter-efficient finetuning with LoRA. Our approach extends the effective context window of a 4k token LLaMA2-7B model to handle up to 128k tokens.
  arXiv  Detail & Related papers  (2024-04-11T17:57:22Z)
• Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts [83.57864140378035]
  This paper proposes a method to cover longer contexts in Open-Domain Question-Answering tasks.<n>It leverages a small encoder language model that effectively encodes contexts, and the encoding applies cross-attention with origin inputs.<n>After fine-tuning, there is improved performance across two held-in datasets, four held-out datasets, and also in two In Context Learning settings.
  arXiv  Detail & Related papers  (2024-04-02T15:10:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.