LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation

• URL: http://arxiv.org/abs/2608.11967v1
• Date: Wed, 12 Aug 2026 11:56:03 GMT
• Title: LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
• Abstract summary: We propose a training framework that formulates reflection as a memory-control policy.<n>LoongReflect combines two complementary signals through a look-ahead, extragradient-style coordination mechanism.<n> Experiments on multi-hop retrieval-augmented generation and mathematical reasoning benchmarks demonstrate consistent improvements.
• Score: 25.555783105371574
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model agents increasingly rely on long-horizon reasoning to solve complex tasks involving planning, tool use, and memory. A critical capability in such settings is reflection: assessing trajectory progress, identifying missing evidence and unreliable intermediate states, and deciding whether to continue, revise, or abandon the current branch. Learning effective reflection, however, is challenging because reflection is performed locally within the current branch, whereas its utility can only be determined by its contribution to the final trajectory outcome. This local-global mismatch makes outcome-based reinforcement learning provide only local, sparse and delayed supervision for reflective decisions. To solve these, we propose LoongReflect, a training framework that formulates reflection as a memory-control policy. The agent operates over a reversible trajectory tree using explicit reflect and backtrack actions. Reflection consolidates verified facts, missing evidence, and branch-specific risks into working memory, while backtracking removes an unreliable branch from the active context and preserves a concise corrective lesson. To learn this policy, LoongReflect combines two complementary signals through a look-ahead, extragradient-style coordination mechanism. A fast channel distills globally informed reflective behavior from a privileged teacher, with supervision restricted to reflection and backtracking tokens. A slow channel optimizes complete trajectories using outcome-based GRPO, aligning local control decisions with final task success. Experiments on multi-hop retrieval-augmented generation and mathematical reasoning benchmarks demonstrate consistent improvements over outcome-only reinforcement learning and self-distillation baselines.

Related papers

• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference [14.39927122740842]
  Reflection merely re-checks established results, it wastes reasoning tokens and increases latency.<n>Most existing reflection steering methods add a label-derived mean-difference direction across preset layers.<n>We propose Reflection Steering, a training-free framework for controlling reflection-associated within LLMs.
  arXiv  Detail & Related papers  (2026-08-26T08:55:28Z)
• ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning [84.29802893743991]
  We show that flawed trajectories can still provide valuable reasoning signals when treated as flawed trajectories to reflect upon.<n>We propose ReflectRL, a lightweight plug-and-play framework that learns from Golden Negative Trajectories during on-policy training.
  arXiv  Detail & Related papers  (2026-08-04T17:40:08Z)
• Group-Reflective Self-Distillation for Agentic Reinforcement Learning [8.041052886037766]
  Group-Reflective Self-Distillation (GRSD) derives capability-aligned and outcome-discriminative guidance from the policy's own verified rollouts.<n>GRSD consistently outperforms competitive baselines and generalizes more effectively to unseen tasks.
  arXiv  Detail & Related papers  (2026-07-30T11:49:46Z)
• HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation [50.53459634301361]
  HERO is a hindsight-enhanced self-distillation framework that uses next environment observations as locally aligned feedback.<n> HERO improves task success and reduces unnecessary turns over environment-feedback-only self-distillation and GRPO.
  arXiv  Detail & Related papers  (2026-06-10T01:35:34Z)
• What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents [70.6980022118038]
  Reinforcement learning can train LLM agents from sparse task rewards, but long-horizon credit assignment remains challenging.<n>Existing methods rely on trajectory-level rewards or proxy signals, without fully leveraging per-step environmental feedback.<n>We introduce SERL, a selective environment-reweighted learning framework.
  arXiv  Detail & Related papers  (2026-05-19T07:00:55Z)
• Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs [63.88783817420284]
  Embodied robots cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials.<n>We introduce Reflective Test-Time Planning, which integrates two modes of reflection: textitreflection-in-action and textitreflection-on-action<n>We also include retrospective reflection, allowing the agent to re-evaluate earlier decisions and perform model updates with hindsight.
  arXiv  Detail & Related papers  (2026-02-24T18:55:18Z)
• RE-TRAC: REcursive TRAjectory Compression for Deep Search Agents [144.5598958575922]
  Re-TRAC is an agentic framework that performs cross-trajectory exploration.<n>We show that Re-TRAC consistently outperforms ReAct by 15-20% on BrowseComp with frontier LLMs.
  arXiv  Detail & Related papers  (2026-02-02T18:58:07Z)
• SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection [14.40651157974557]
  SAMULE is a new framework for self-learning agents powered by a retrospective language model that is trained based on Multi-Level Reflection Synthesis.<n>It first synthesizes high-quality reflections across three complementary levels: Single-Trajectory Learning (micro-level) for detailed error correction; Intra-Task Learning (meso-level) to build error across multiple trials of the same task, and Inter-Task Learning (macro-level) to extract transferable insights based on same typed errors from diverse task failures.
  arXiv  Detail & Related papers  (2025-09-24T21:02:15Z)
• Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning [87.7836502955847]
  We propose a novel self-rewarding reinforcement learning framework to enhance Large Language Model (LLM) reasoning.<n>Our key insight is that correct responses often exhibit consistent trajectory patterns in terms of model likelihood.<n>We introduce CoVo, an intrinsic reward mechanism that integrates Consistency and Volatility via a robust vector-space aggregation strategy.
  arXiv  Detail & Related papers  (2025-06-10T12:40:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.