Closing the Loop on Latent Reasoning via Test-Time Reconstruction

• URL: http://arxiv.org/abs/2606.06252v1
• Date: Thu, 04 Jun 2026 14:54:40 GMT
• Title: Closing the Loop on Latent Reasoning via Test-Time Reconstruction
• Abstract summary: Recent work moves intermediate reasoning from natural-language traces into latent or cache-level representations.<n>We propose ReLAT, a self-supervised test-time training method that closes this loop using the query itself as the reference.<n>We show that ReLAT consistently improves over single-model inference, text-based collaboration, open-loop latent collaboration, and alternative test-time training objectives.
• Score: 45.08180971427891
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work moves intermediate reasoning from natural-language traces into latent or cache-level representations to reduce token overhead and avoid a discrete communication bottleneck. However, this shift also removes a key advantage of textual reasoning: intermediate states are no longer inspectable, making it difficult to determine whether a latent state still preserves the constraints of the original query. As a result, latent reasoning typically operates in an open loop, where a latent state is produced and consumed without an input-anchored fidelity check. We propose ReLAT (Reconstruction-Guided Latent Reasoning At Test Time), a self-supervised test-time training method that closes this loop using the query itself as the reference. Our key observation is that if a latent state faithfully represents a query, the query should be recoverable from it; if the query cannot be recovered, the latent state has lost task-relevant information. ReLAT operationalizes this principle by constructing a differentiable Question -> Latent Thought -> Question cycle and optimizing query reconstruction loss through the latent thought before answer generation. This anchors opaque latent computation to the problem specification it is supposed to represent. Across mathematical reasoning, knowledge QA, and code generation benchmarks on the Qwen family, ReLAT consistently improves over single-model inference, text-based collaboration, open-loop latent collaboration, and alternative test-time training objectives. On Qwen3-8B, ReLAT raises AIME 2024 accuracy from 56.7% to 73.3%, a 16.6-point gain over the strongest open-loop latent baseline.

Related papers

• WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models [51.88075626243057]
  We propose WhisperRec, an efficient latent reasoning framework for foundation recommendation models (FRMs)<n>WhisperRec compresses teacher-generated Chain-of-Thought (CoT) into learnable latent reasoning tokens.<n>We show that WhisperRec consistently outperforms explicit-CoT methods and conventional baselines.
  arXiv  Detail & Related papers  (2026-07-29T08:48:35Z)
• Latent Thought Flow: Efficient Latent Reasoning in Large Language Models [23.52673819075993]
  We propose Latent Thought Flow (LTF), which models reasoning as variable-length continuous trajectories and trains a sampler to match a reward-induced posterior over answer quality and computation cost.<n>Experiments under finetuning and transfer learning settings show that LTF outperforms explicit CoT and latent reasoning baselines, improving accuracy by 9.5% and reducing reasoning length by 27.2% on average.
  arXiv  Detail & Related papers  (2026-06-15T05:02:46Z)
• CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning [22.944748148277146]
  CopT is a reformulated reasoning pipeline that reverses the usual order of thinking and answering.<n>CopT elicits a draft answer and then invokes subsequent on-policy thinking conditioned on its own draft answer.<n>CopT improves peak accuracy by up to 23% and reduces token usage by up to 57% at comparable or higher accuracy.
  arXiv  Detail & Related papers  (2026-05-19T16:28:53Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards [61.52322047892064]
  Test-Time Scaling (TTS) has significantly enhanced the capabilities of Large Reasoning Models (LRMs)<n>We observe that LRMs frequently conduct repetitive self-verification without revision even after obtaining the final answer during the reasoning process.<n>We propose Anchor-based Process Reward (APR), a structure-aware reward shaping method that localizes the reasoning anchor and penalizes exclusively the post-anchor AST.
  arXiv  Detail & Related papers  (2026-01-31T14:53:20Z)
• QUARK: Robust Retrieval under Non-Faithful Queries via Query-Anchored Aggregation [2.505352949111876]
  QUARK is a training-free framework for robust retrieval under non-faithful queries.<n>The design enables QUARK to improve recall and ranking quality without sacrificing robustness.
  arXiv  Detail & Related papers  (2026-01-28T21:14:49Z)
• RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability [12.67288560758937]
  We propose RaCoT (Retrieval-aware Contrastive-of-Thought), a novel framework that shifts contrastive thinking to the pre-retrieval stage.<n>RaCoT guides the model to proactively focus on the critical details that determine answer divergence"
  arXiv  Detail & Related papers  (2025-10-26T15:06:44Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Reading Between the Timelines: RAG for Answering Diachronic Questions [8.969698902720799]
  We propose a new framework that fundamentally redesigns the RAG pipeline to infuse temporal logic.<n>Our approach yields substantial gains in answer accuracy, surpassing standard RAG implementations by 13% to 27%.<n>This work provides a validated pathway toward RAG systems capable of performing the nuanced, evolutionary analysis required for complex, real-world questions.
  arXiv  Detail & Related papers  (2025-07-21T05:19:41Z)
• Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer [0.8738725605667471]
  Chain-of-thought (CoT) reasoning has enabled transformer-based language models to excel at complex mathematics and multi-step planning.<n>In standard decoder-only architectures, these reasoning steps are externalized in natural language, improving interpretability at the cost of efficiency.<n>We investigate whether such reasoning structures emerge in Huginn-3.5B, a depth-recurrent Transformer that reuses layers at inference time without increasing parameter count.
  arXiv  Detail & Related papers  (2025-07-02T23:35:21Z)
• Logically Consistent Loss for Visual Question Answering [66.83963844316561]
  The current advancement in neural-network based Visual Question Answering (VQA) cannot ensure such consistency due to identically distribution (i.i.d.) assumption. We propose a new model-agnostic logic constraint to tackle this issue by formulating a logically consistent loss in the multi-task learning framework. Experiments confirm that the proposed loss formulae and introduction of hybrid-batch leads to more consistency as well as better performance.
  arXiv  Detail & Related papers  (2020-11-19T20:31:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.