Spatially Generalizable Mobile Manipulation via Adaptive Experience Selection and Dynamic Imagination

• URL: http://arxiv.org/abs/2601.14649v1
• Date: Wed, 21 Jan 2026 04:43:49 GMT
• Title: Spatially Generalizable Mobile Manipulation via Adaptive Experience Selection and Dynamic Imagination
• Abstract summary: Mobile Manipulation involves long-horizon decision-making over multi-stage compositions of heterogeneous skills.<n>Existing MM methods face two key limitations: (i) low sample efficiency, due to ineffective use of redundant data generated during long-term interactions.<n>In this paper, we address these challenges through Adaptive Experience Selection (AES) and model-based dynamic imagination.
• Score: 23.225672159796815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mobile Manipulation (MM) involves long-horizon decision-making over multi-stage compositions of heterogeneous skills, such as navigation and picking up objects. Despite recent progress, existing MM methods still face two key limitations: (i) low sample efficiency, due to ineffective use of redundant data generated during long-term MM interactions; and (ii) poor spatial generalization, as policies trained on specific tasks struggle to transfer to new spatial layouts without additional training. In this paper, we address these challenges through Adaptive Experience Selection (AES) and model-based dynamic imagination. In particular, AES makes MM agents pay more attention to critical experience fragments in long trajectories that affect task success, improving skill chain learning and mitigating skill forgetting. Based on AES, a Recurrent State-Space Model (RSSM) is introduced for Model-Predictive Forward Planning (MPFP) by capturing the coupled dynamics between the mobile base and the manipulator and imagining the dynamics of future manipulations. RSSM-based MPFP can reinforce MM skill learning on the current task while enabling effective generalization to new spatial layouts. Comparative studies across different experimental configurations demonstrate that our method significantly outperforms existing MM policies. Real-world experiments further validate the feasibility and practicality of our method.

Related papers

• MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models [55.732137666068446]
  Universal multimodal embedding (UME) increasingly demands encoder's capacity for handling a broad range of tasks and modalities.<n>MoEMB scales UME along the expert axis through mixture-of-experts (MoE)<n>MoEMB surpasses TTE-based methods with >4x active parameters, using significantly less computes.
  arXiv  Detail & Related papers  (2026-09-08T12:29:50Z)
• CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection [33.94199994115719]
  CL-Anomaly is a parameter-efficient fine-tuning framework for continuous learning for anomaly detection.<n>PrivLoRA physically isolates task-specific subspaces in the parameter space to prevent semantic entanglement of anomaly knowledge.<n> Layer-Adaptive Shared Experts maintain cross-task representations within a unified feature space.
  arXiv  Detail & Related papers  (2026-07-03T03:53:58Z)
• LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective [7.2941149985927645]
  This paper presents an architectural perspective on the Lifelong Mixture of Dynamic Experts (textitLiMoDE)<n>Specifically, a dynamic MoE structure is first proposed in the multi-task pre-training stage to learn prior knowledge.<n>In the task adaptation stage, we design a lifelong MoE adaptation mechanism % (LiMoEAM) that learns lifelong experts and dynamically combines them with frozen ones for new tasks.
  arXiv  Detail & Related papers  (2026-06-24T13:18:43Z)
• Delve into the Applicability of Advanced Optimizers for Multi-Task Learning [70.80216936064654]
  Multi-Task Learning (MTL) is a machine learning problem that has seen extensive development over the past decade.<n>We propose APT (Applicability of advanced oPizers), a framework featuring a simple adaptive momentum mechanism designed to balance the strengths between advanceds and MTL.
  arXiv  Detail & Related papers  (2026-04-10T04:15:31Z)
• Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies [13.826293542795938]
  Diffusion-based policies have recently shown strong results in robot manipulation, but their extension to multi-task scenarios is hindered by the high cost of scaling model size and demonstrations.<n>We introduce Skill Mixture-of-Experts Policy (SMP), a diffusion-based mixture-of-experts policy that learns a compact skill basis and uses sticky routing to compose actions from a small, task-relevant subset of experts at each step.<n>We validate SMP in simulation and on a real dual-arm platform with multi-task learning and transfer learning tasks, where SMP achieves higher success rates and markedly lower inference cost than large diffusion baselines.
  arXiv  Detail & Related papers  (2026-01-29T04:17:56Z)
• Self-Controlled Dynamic Expansion Model for Continual Learning [10.447232167638816]
  This paper introduces an innovative Self-Controlled Dynamic Expansion Model (SCDEM)<n>SCDEM orchestrates multiple trainable pre-trained ViT backbones to furnish diverse and semantically enriched representations.<n>An extensive series of experiments have been conducted to evaluate the proposed methodology's efficacy.
  arXiv  Detail & Related papers  (2025-04-14T15:22:51Z)
• MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation [62.854649499866774]
  Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotics manipulation and navigation.<n>We propose a novel multi-agent LLM framework that distributes high-level planning and low-level control code generation across specialized LLM agents.<n>We evaluate our approach on nine RLBench tasks, including long-horizon tasks, and demonstrate its ability to solve robotics manipulation in a zero-shot setting.
  arXiv  Detail & Related papers  (2024-11-26T17:53:44Z)
• Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula [23.071384759427072]
  State space models (SSMs) offer advantages over Transformers but struggle with tasks requiring long-range in-context retrieval-like text copying, associative recall, and question answering over long contexts.<n>We propose a novel training procedure, Birdie, that significantly enhances the in-context retrieval capabilities of SSMs without altering their architecture.
  arXiv  Detail & Related papers  (2024-11-01T21:01:13Z)
• On-the-fly Modulation for Balanced Multimodal Learning [53.616094855778954]
  Multimodal learning is expected to boost model performance by integrating information from different modalities. The widely-used joint training strategy leads to imbalanced and under-optimized uni-modal representations. We propose On-the-fly Prediction Modulation (OPM) and On-the-fly Gradient Modulation (OGM) strategies to modulate the optimization of each modality.
  arXiv  Detail & Related papers  (2024-10-15T13:15:50Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• T-REX: Mixture-of-Rank-One-Experts with Semantic-aware Intuition for Multi-task Large Language Model Finetuning [31.276142111455847]
  Large language models (LLMs) encounter significant adaptation challenges in diverse multitask finetuning.<n>We design a novel framework, mixunderlinetextbfTureunderlinetextbf-of-underlinetextbfRank-onunderlinetextbfE-eunderlinetextbfXper ts (textttT-REX)<n>Rank-1 experts enable a mix-and-match mechanism to quadratically expand the vector subspace of experts with linear parameter overheads, achieving approximate error reduction with optimal
  arXiv  Detail & Related papers  (2024-04-13T12:14:58Z)
• Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving [103.745551954983]
  In this paper, we investigate the transfer performance of various types of self-supervised methods, including MoCo and SimCLR, on three downstream tasks. We find that their performances are sub-optimal or even lag far behind the single-task baseline. We propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training.
  arXiv  Detail & Related papers  (2022-09-19T12:15:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.