Decocted Experience Improves Test-Time Inference in LLM Agents

• URL: http://arxiv.org/abs/2604.04373v1
• Date: Mon, 06 Apr 2026 02:49:32 GMT
• Title: Decocted Experience Improves Test-Time Inference in LLM Agents
• Abstract summary: We show that effective context construction critically depends on emphdecocted experience<n>We identify emphdecocted experience as a key mechanism for effective context construction.<n>We validate our findings across reasoning and agentic tasks, including math reasoning, web browsing, and software engineering.
• Score: 30.56835805919089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is growing interest in improving LLMs without updating model parameters. One well-established direction is test-time scaling, where increased inference-time computation (e.g., longer reasoning, sampling, or search) is used to improve performance. However, for complex reasoning and agentic tasks, naively scaling test-time compute can substantially increase cost and still lead to wasted budget on suboptimal exploration. In this paper, we explore \emph{context} as a complementary scaling axis for improving LLM performance, and systematically study how to construct better inputs that guide reasoning through \emph{experience}. We show that effective context construction critically depends on \emph{decocted experience}. We present a detailed analysis of experience-augmented agents, studying how to derive context from experience, how performance scales with accumulated experience, what characterizes good context, and which data structures best support context construction. We identify \emph{decocted experience} as a key mechanism for effective context construction: extracting essence from experience, organizing it coherently, and retrieving salient information to build effective context. We validate our findings across reasoning and agentic tasks, including math reasoning, web browsing, and software engineering.

Related papers

• CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs [50.189987475377656]
  We propose CARE, a competence-aware reward shaping framework for adaptive reasoning length optimization in multimodal reasoning.<n> CARE maintains a smoothed competence estimate via an exponential moving average of pass rates, and uses it to route training into progressive stages.<n>Experiments on multiple video reasoning and general video understanding benchmarks demonstrate that CARE consistently improves reasoning accuracy, stabilizes reinforcement learning, and significantly enhances token efficiency.
  arXiv  Detail & Related papers  (2026-06-18T08:28:26Z)
• Prompt Quality and Pull Request Outcomes: A Stage-Based Empirical Study of LLM-Assisted Development [0.0]
  We analyze 265 developer-ChatGPT interactions derived from self-admitted ChatGPT usage in pull requests.<n>We operationalize prompt structure using three dimensions: Context, Specificity, and Verification.<n>Our findings show that prompt characteristics exert distinct, stage-dependent effects across AI-assisted software engineering.
  arXiv  Detail & Related papers  (2026-06-17T22:50:22Z)
• Rethinking Experience Utilization in Self-Evolving Language Model Agents [51.10420305280499]
  Self-evolving agents improve by accumulating and reusing experience from past interactions.<n>This paper studies experience utilization as a critical design dimension of self-evolving agents.
  arXiv  Detail & Related papers  (2026-05-08T02:48:49Z)
• CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection [17.486111179784757]
  We propose a generative context augmentation framework using Contrastive Learning of Experience via Agentic Reflection (CLEAR)<n>CLEAR first employs a reflection agent to perform contrastive analysis over past execution trajectories and summarize useful context for each observed task.<n>We further optimize CAM using reinforcement learning, where the reward signal is obtained by running the task execution agent.
  arXiv  Detail & Related papers  (2026-04-08T18:26:59Z)
• Retrieval-Augmented LLM Agents: Learning to Learn from Experience [16.248836438253814]
  We study how to train retrieval-augmented LLM agents to leverage retrieved trajectories in-context.<n>We establish a robust supervised fine-tuning (SFT) recipe using LoRA that outperforms several state-of-the-art agent training pipelines.<n>Our results demonstrate that this combined approach significantly improves generalization to unseen tasks.
  arXiv  Detail & Related papers  (2026-03-18T20:45:04Z)
• Predicting Task Performance with Context-aware Scaling Laws [56.6850444554434]
  We propose a straightforward, interpretable framework that jointly models downstream performance as a function of the training compute and the provided context.<n>We empirically validate our framework by fitting it on the observed downstream performance of extended-context variants of Llama-2-7B and Llama-2-13B.<n>Our results demonstrate that our framework accurately models in-distribution downstream performance, generalizes across three orders of magnitude in training compute, and reliably extrapolates performance as the amount of context increases.
  arXiv  Detail & Related papers  (2025-10-16T17:35:18Z)
• Pushing LLMs to Their Logical Reasoning Bound: The Role of Data Reasoning Intensity [59.27594125465172]
  We introduce Data Reasoning Intensity (DRI), a novel metric that quantifies the latent logical reasoning complexity of samples.<n>We then introduce a re-cognizing optimization strategy that systematically enhances the logical reasoning intensity of training data.
  arXiv  Detail & Related papers  (2025-09-29T14:20:04Z)
• Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning [68.89572566071575]
  Tool-Integrated Reasoning (TIR) enables large language models (LLMs) to improve their internal reasoning ability by integrating external tools.<n>We propose Tool-Light, a framework designed to encourage LLMs to perform TIR efficiently and accurately.<n> Experimental results on 10 datasets demonstrate the effectiveness of Tool-Light.
  arXiv  Detail & Related papers  (2025-09-27T12:53:37Z)
• Improving Context Fidelity via Native Retrieval-Augmented Reasoning [35.50952279309109]
  Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information.<n>We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities.<n>Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain.
  arXiv  Detail & Related papers  (2025-09-17T04:28:07Z)
• Improving LLM Agent Planning with In-Context Learning via Atomic Fact Augmentation and Lookahead Search [48.348209577994865]
  Large Language Models (LLMs) are increasingly capable but often require significant guidance or extensive interaction history to perform effectively in complex, interactive environments.<n>We introduce a novel LLM agent framework that enhances planning capabilities through in-context learning.<n>Our agent learns to extract task-critical atomic facts'' from its interaction trajectories.
  arXiv  Detail & Related papers  (2025-06-10T18:36:31Z)
• Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experience [8.270615745804557]
  Large language models (LLMs) achieve strong performance on plain text tasks but underperform on structured data like tables and databases.<n>We introduce Contrastive Retrieval-Augmented Generation on Experience (CoRE), a framework that builds experience memory representations.<n> Experiments show CoRE significantly improves performance, achieving average gains of 3.44% and 4.24%, with up to 17.2% on challenging tasks.
  arXiv  Detail & Related papers  (2025-06-01T05:22:00Z)
• Eliciting Causal Abilities in Large Language Models for Reasoning Tasks [14.512834333917414]
  We introduce the Self-Causal Instruction Enhancement (SCIE) method, which enables LLMs to generate high-quality, low-quantity observational data.<n>In SCIE, the instructions are treated as the treatment, and textual features are used to process natural language.<n>Our method effectively generates instructions that enhance reasoning performance with reduced training cost of prompts.
  arXiv  Detail & Related papers  (2024-12-19T17:03:02Z)
• CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing [70.25689961697523]
  We propose a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection. Our work bridges the gap between existing sequential reasoning paradigms and validates the effectiveness of leveraging cross-task experiences.
  arXiv  Detail & Related papers  (2024-10-22T03:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.