Dynamic Latent Routing

• URL: http://arxiv.org/abs/2605.14323v1
• Date: Thu, 14 May 2026 03:35:46 GMT
• Title: Dynamic Latent Routing
• Abstract summary: We introduce General Dijkstra Search (GDS), and prove that globally optimal goal-reaching policies can be recovered through temporal composition of intermediate optimal sub-policies.<n>Motivated by the "search, select, update" principle underlying GDS, we propose Dynamic Latent Routing (DLR), a language-model post-training method that jointly learns discrete latent codes, routing policies, and model parameters through dynamic search in a single training stage.
• Score: 3.2461838619378702
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We investigate the temporal concatenation of sub-policies in Markov Decision Processes (MDP) with time-varying reward functions. We introduce General Dijkstra Search (GDS), and prove that globally optimal goal-reaching policies can be recovered through temporal composition of intermediate optimal sub-policies. Motivated by the "search, select, update" principle underlying GDS, we propose Dynamic Latent Routing (DLR), a language-model post-training method that jointly learns discrete latent codes, routing policies, and model parameters through dynamic search in a single training stage. In low-data fine-tuning settings, DLR matches or outperforms supervised fine-tuning across four datasets and six models, achieving a mean gain of +6.6 percentage points, while prior discrete-latent baselines consistently underperform SFT. Mechanistic analyses and targeted code ablations show that DLR learns structured routing behaviors with distinct causal roles.

Related papers

• LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents [51.74109282213905]
  Post-training strategies are dataset-dependent and reveal a recurring empirical pattern.<n>Regularization parameters oscillate in response to shifting training dynamics.<n>We use a system where LLM agents search over training trajectories via tree search.
  arXiv  Detail & Related papers  (2026-06-16T18:33:08Z)
• Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planning [21.105233947788104]
  We introduce a model-based hierarchical framework for sequential decision-making.<n>We learn a subgoal-conditioned budget policy jointly with the world model to support context-aware resource allocation.<n>Across challenging SSCO benchmarks, our method outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-05-16T16:04:11Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning [20.48365890565577]
  We propose a novel meta-RLVR algorithm that dynamically selects training data at each step based on the average response length.<n>We evaluate LSPO across multiple base models and datasets, demonstrating that it consistently improves learning effectiveness.
  arXiv  Detail & Related papers  (2025-10-01T20:57:22Z)
• DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTS [28.828541350757714]
  This paper proposes Dynamically Adaptive MCTS-based Reasoning (DAMR) for Knowledge Graph Question Answering (KGQA)<n>DAMR integrates Monte Carlo Tree Search (MCTS) with adaptive path evaluation to enable context-aware KGQA.<n>Experiments on multiple KGQA benchmarks show DAMR significantly outperforms SOTA methods.
  arXiv  Detail & Related papers  (2025-08-01T15:38:21Z)
• Reinforce LLM Reasoning through Multi-Agent Reflection [8.088795955922656]
  We introduce DPSDP, a reinforcement learning algorithm that trains an actor-critic LLM system to iteratively refine answers via direct preference learning on self-generated data.<n>Theoretically, DPSDP can match the performance of any policy within the training distribution.<n>For example, on benchmark MATH 500, majority voting over five refinement steps increases first-turn accuracy from 58.2% to 63.2% with Ministral-based models.
  arXiv  Detail & Related papers  (2025-06-10T02:43:47Z)
• Prior-Guided Diffusion Planning for Offline Reinforcement Learning [5.819784482811376]
  Prior Guidance (PG) is a novel guided sampling framework that replaces the standard Gaussian prior of a behavior-cloned diffusion model.<n>We present an efficient training strategy that applies behavior regularization in latent space, and empirically demonstrate that PG outperforms state-the-art diffusion policies and planners across diverse long-horizon offline RL benchmarks.
  arXiv  Detail & Related papers  (2025-05-16T05:39:02Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Statistically Efficient Variance Reduction with Double Policy Estimation for Off-Policy Evaluation in Sequence-Modeled Reinforcement Learning [53.97273491846883]
  We propose DPE: an RL algorithm that blends offline sequence modeling and offline reinforcement learning with Double Policy Estimation. We validate our method in multiple tasks of OpenAI Gym with D4RL benchmarks.
  arXiv  Detail & Related papers  (2023-08-28T20:46:07Z)
• Reparameterized Policy Learning for Multimodal Trajectory Optimization [61.13228961771765]
  We investigate the challenge of parametrizing policies for reinforcement learning in high-dimensional continuous action spaces. We propose a principled framework that models the continuous RL policy as a generative model of optimal trajectories. We present a practical model-based RL method, which leverages the multimodal policy parameterization and learned world model.
  arXiv  Detail & Related papers  (2023-07-20T09:05:46Z)
• RoMA: Robust Model Adaptation for Offline Model-based Optimization [115.02677045518692]
  We consider the problem of searching an input maximizing a black-box objective function given a static dataset of input-output queries. A popular approach to solving this problem is maintaining a proxy model that approximates the true objective function. Here, the main challenge is how to avoid adversarially optimized inputs during the search.
  arXiv  Detail & Related papers  (2021-10-27T05:37:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.