MPCoT: Reward-Guided Multi-Path Latent Reasoning for Test-Time Scalable Vision-Language-Action

• URL: http://arxiv.org/abs/2606.06245v1
• Date: Thu, 04 Jun 2026 14:48:44 GMT
• Title: MPCoT: Reward-Guided Multi-Path Latent Reasoning for Test-Time Scalable Vision-Language-Action
• Abstract summary: We propose MPCoT, a reward-guided multi-path latent reasoning framework.<n>MPCoT initializes $M$ hypotheses, refines them for K weight-tied steps, and softly aggregates them before action decoding.<n>It improves long-horizon performance, with ablations confirming depth-width effects, confidence-weighted aggregation, and reward-guided path supervision.
• Score: 6.960597813771435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) policies remain brittle in long-horizon and high-uncertainty control, where one-pass action decoding provides limited inference-time deliberation. Explicit chain-of-thought can increase reasoning depth, but introduces token latency and an indirect text-to-action interface. We propose MPCoT, a reward-guided multi-path latent reasoning framework that initializes $M$ hypotheses, refines them for K weight-tied steps, and softly aggregates them before action decoding. A training-only path-preference objective evaluates candidate action branches with expert-action consistency, world-model/VLM-based progress, and success feedback to align the latent path scorer with downstream execution quality. MPCoT preserves the original 8-step action interface, generates zero reasoning tokens, and exposes configurable inference controls (K,M). Under matched protocols on LIBERO and CALVIN, MPCoT improves long-horizon performance, with ablations confirming depth-width effects, confidence-weighted aggregation, and reward-guided path supervision.

Related papers

• Structural Process Supervision for Latent Chain-of-Thought Reasoning [45.1922274576773]
  Prototype-Mediated Process Supervision (PMPS) projects latent embeddings and explicit chain-of-thought embeddings into a shared prototype space.<n>PMPS compresses output token length to under 50% of explicit CoT on GSM8K-Aug.<n>On larger models and a more challenging task, PMPS consistently attains the highest accuracy among all latent reasoning methods with comparable output length.
  arXiv  Detail & Related papers  (2026-09-09T09:17:46Z)
• TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
  Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
  arXiv  Detail & Related papers  (2026-06-09T17:16:03Z)
• TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization [18.977861031175756]
  TARPO is a pure RL framework that switches between discrete token generation and continuous latent reasoning at each step.<n> TARPO consistently outperforms existing explicit and latent reasoning RL baselines across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-06-04T08:30:53Z)
• TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition [62.56752617853322]
  Outcome-based rewards provide only sparse feedback, while trajectory-supervised rewards depend on annotated reference solutions.<n>We propose TIER: Trajectory-Invariant Execution Rewards, a reward framework that derives supervision directly from function schemas and runtime execution.
  arXiv  Detail & Related papers  (2026-05-16T03:47:26Z)
• CATP: Confidence-Aware Token Pruning for Camouflaged Object Detection [17.488230092440144]
  We propose a hierarchical Confidence-Aware Token Pruning framework (CATP) tailored for Camouflaged Object Detection (COD)<n>Our approach hierarchically identifies and discards easily distinguishable tokens from both background and object interiors, focusing computations on critical boundary tokens.<n>To compensate for information loss from pruning, we introduce a dual-path feature compensation mechanism that aggregates contextual knowledge from pruned tokens into enriched features.
  arXiv  Detail & Related papers  (2026-04-18T06:03:18Z)
• PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs [0.0]
  Large Language Models (LLMs) often struggle with maintaining coherent multi-step reasoning traces.<n>This work introduces a quantum-inspired approach to address the challenge by incorporating a fidelity-based reward.<n>Results show that the proposed quantum-inspired approach offers significant improvements over supervised, contrastive, and pretrained baseline approaches.
  arXiv  Detail & Related papers  (2025-09-24T13:29:53Z)
• Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning [0.13192560874022083]
  We present Vis-CoT, a human-in-the-loop framework that converts linear chain-of-thought text into an interactive reasoning graph.<n>Users can visualize the logical flow, identify flawed steps, and intervene by pruning incorrect paths and grafting new, user-defined premises.<n> Vis-CoT improves final-answer accuracy by up to 24 percentage points over non-interactive baselines.
  arXiv  Detail & Related papers  (2025-09-01T12:09:43Z)
• ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning [64.93140713419561]
  Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs.<n>Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection.<n>We introduce ConCISE, a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient.
  arXiv  Detail & Related papers  (2025-05-08T01:40:40Z)
• Test-Time Alignment for Large Language Models via Textual Model Predictive Control [63.508812485566374]
  Textual Model Predictive Control (TMPC) is a novel predictive planning framework adapted for aligning Large Language Models at inference time.<n>TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis.<n>Results demonstrate that TMPC consistently improves performance, highlighting the generality.
  arXiv  Detail & Related papers  (2025-02-28T07:24:33Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)
• Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation [85.22775182688798]
  This work proposes a novel, flexible, and accurate refinement module called Alpha-Refine. It can significantly improve the base trackers' box estimation quality. Experiments on TrackingNet, LaSOT, GOT-10K, and VOT 2020 benchmarks show that our approach significantly improves the base trackers' performance with little extra latency.
  arXiv  Detail & Related papers  (2020-12-12T13:33:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.