LAR-MoE: Latent-Aligned Routing for Mixture of Experts in Robotic Imitation Learning

• URL: http://arxiv.org/abs/2603.08476v1
• Date: Mon, 09 Mar 2026 15:10:31 GMT
• Title: LAR-MoE: Latent-Aligned Routing for Mixture of Experts in Robotic Imitation Learning
• Abstract summary: We introduce Latent-Aligned Routing for Mixture of Experts (LAR-MoE), a two-stage framework that decouples unsupervised skill discovery from policy learning.<n>In pre-training, we learn a joint latent representation between observations and future actions through student-teacher co-training.<n>In a post-training stage, the expert routing is regularized to follow the structure of the learned latent space, preventing expert collapse.
• Score: 1.7272266547709354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imitation learning enables robots to acquire manipulation skills from demonstrations, yet deploying a policy across tasks with heterogeneous dynamics remains challenging, as models tend to average over distinct behavioral modes present in the demonstrations. Mixture-of-Experts (MoE) architectures address this by activating specialized subnetworks, but requires meaningful skill decompositions for expert routing. We introduce Latent-Aligned Routing for Mixture of Experts (LAR-MoE), a two-stage framework that decouples unsupervised skill discovery from policy learning. In pre-training, we learn a joint latent representation between observations and future actions through student-teacher co-training. In a post-training stage, the expert routing is regularized to follow the structure of the learned latent space, preventing expert collapse while maintaining parameter efficiency. We evaluate LAR-MoE in simulation and on hardware. On the LIBERO benchmark, our method achieves a 95.2% average success rate with 150M parameters. On a surgical bowel grasping and retraction task, LAR-MoE matches a supervised MoE baseline without requiring any phase annotations, and transfers zero-shot to ex vivo porcine tissue. Our findings suggest that latent-aligned routing provides a principled alternative to supervised skill decomposition, enabling structured expert specialization from unlabeled demonstrations.

Related papers

• Emergent Compositional Skills in Mixture-of-Experts VLAs [4.404201871360367]
  We ask whether a VLA trained with a simplified Mixture-of-Experts (MoE) action head can emergently learn to decompose tasks into reusable, interpretable primitives.<n>We find that learned experts are heavily reused across tasks and consistently correspond to qualitatively distinct low-level behaviors.
  arXiv  Detail & Related papers  (2026-07-22T22:36:52Z)
• Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models [50.34585122347149]
  We introduce Expert Tying, an architectural modification that shares expert parameters across consecutive transformer layers.<n>We evaluate this approach across common, state-of-the-art architectures, including OLMoE, Qwen3, and DeepSeek-style MoEs.
  arXiv  Detail & Related papers  (2026-06-15T15:08:09Z)
• Learning Multi-Modal Trajectory Policies for Data-Efficient Robotic Manipulation [21.99077052779724]
  MATE (Multi-Modal MoE) is a novel trajectory prediction framework built upon MoE.<n>We introduce a Multi-Modal MoE architecture to achieve fine-grained sub-modal cosine, and design a cross-modal router for stable and scale-invariant expert assignment.<n>Experiments on robotic ping-pong show that our MATE consistently outperforms prior work under data scarcity.
  arXiv  Detail & Related papers  (2026-05-31T06:35:10Z)
• SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning [83.66308307152808]
  We propose StAbilized Mixture-of-Experts (SAME) for Multimodal Continual Instruction Tuning (MCIT)<n>SAME stabilizes expert selection by decomposing routing dynamics into subspaces and updating only task-relevant directions.<n>It also introduces adaptive expert activation to freeze selected experts during training, reducing redundant and cross-task interference.
  arXiv  Detail & Related papers  (2026-02-02T11:47:06Z)
• Learning Diverse Skills for Behavior Models with Mixture of Experts [12.076288904201418]
  We propose to learn diverse skills for behavior models with Mixture of Experts, referred to as Di-BM.<n>Our approach is plug-and-play and can be seamlessly integrated into standard imitation learning methods.<n>Experiments on multiple real-world robotic manipulation tasks demonstrate that Di-BM significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-01-18T13:14:32Z)
• Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance [79.21541758879012]
  Mixture-of-Experts (MoE) has emerged as a powerful paradigm for scaling model capacity while preserving computational efficiency.<n>We present ProMoE, an MoE framework featuring a two-step router with explicit routing guidance that promotes expert specialization.
  arXiv  Detail & Related papers  (2025-10-28T17:59:02Z)
• Guided by the Experts: Provable Feature Learning Dynamic of Soft-Routed Mixture-of-Experts [11.437368205968573]
  This paper advances MoE theory by providing convergence guarantees for joint training of soft-routed MoE models with non-linear routers and experts.<n>We show that a post-training pruning can effectively eliminate redundant neurons, followed by a provably convergent fine-tuning process that reaches global optimality.
  arXiv  Detail & Related papers  (2025-10-08T16:40:31Z)
• Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms [55.1784306456972]
  Mixture-of-Experts (MoE) architectures have emerged as a promising direction, offering efficiency and scalability by activating only a subset of parameters during inference.<n>We use an internal metric to investigate the mechanisms of MoE architecture by explicitly incorporating routing mechanisms and analyzing expert-level behaviors.<n>We uncover several findings: (1) neuron utilization decreases as models evolve, reflecting stronger generalization; (2) training exhibits a dynamic trajectory, where benchmark performance alone provides limited signal; (3) task completion emerges from collaborative contributions of multiple experts, with shared experts driving concentration; and (4) activation patterns at the neuron level provide a fine-grained proxy for data diversity.
  arXiv  Detail & Related papers  (2025-09-28T15:13:38Z)
• MoTE: Mixture of Task-specific Experts for Pre-Trained ModelBased Class-incremental Learning [39.892628170627496]
  Class-incremental learning (CIL) requires deep learning models to continuously acquire new knowledge from streaming data.<n> prompt-based approaches suffer from prompt overwriting, while adapter-based methods face challenges such as dimensional misalignment between tasks.<n>We propose a mixture of task-specific experts (MoTE) framework that effectively mitigates the miscalibration caused by inconsistent output dimensions.
  arXiv  Detail & Related papers  (2025-05-21T03:06:10Z)
• MoE-Loco: Mixture of Experts for Multitask Locomotion [52.04025933292957]
  We present MoE-Loco, a framework for multitask locomotion for legged robots.<n>Our method enables a single policy to handle diverse terrains, while supporting quadrupedal and bipedal gaits.
  arXiv  Detail & Related papers  (2025-03-11T15:53:54Z)
• Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training [73.90260246781435]
  We present Lory, the first approach that scales such architectures to autoregressive language model pre-training. We show significant performance gains over parameter-matched dense models on both perplexity and a variety of downstream tasks. Despite segment-level routing, Lory models achieve competitive performance compared to state-of-the-art MoE models with token-level routing.
  arXiv  Detail & Related papers  (2024-05-06T03:06:33Z)
• Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL) [1.0057319866872687]
  This paper presents a method called Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL) CMZ-DRIL uses reinforcement learning to minimize uncertainty among an ensemble of agents trained to model the expert demonstrations. As demonstrated in a waypoint-navigation environment and in two MuJoCo environments, CMZ-DRIL can generate performant agents that behave more similarly to the expert.
  arXiv  Detail & Related papers  (2024-03-02T01:40:37Z)
• Conditional Neural Expert Processes for Learning Movement Primitives from Demonstration [1.9336815376402723]
  Conditional Neural Expert Processes (CNEP) learns to assign demonstrations from different modes to distinct expert networks. CNEP does not require supervision on which mode the trajectories belong to. Our system is capable of on-the-fly adaptation to environmental changes via an online conditioning mechanism.
  arXiv  Detail & Related papers  (2024-02-13T12:52:02Z)
• Imitation from Observation With Bootstrapped Contrastive Learning [12.048166025000976]
  Imitation from observation (IfO) is a learning paradigm that consists of training autonomous agents in a Markov Decision Process. We present BootIfOL, an IfO algorithm that aims to learn a reward function that takes an agent trajectory and compares it to an expert. We evaluate our approach on a variety of control tasks showing that we can train effective policies using a limited number of demonstrative trajectories.
  arXiv  Detail & Related papers  (2023-02-13T17:32:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.