Perceptive Hierarchical-Task MPC for Sequential Mobile Manipulation in Unstructured Semi-Static Environments

• URL: http://arxiv.org/abs/2603.10227v1
• Date: Tue, 10 Mar 2026 20:55:30 GMT
• Title: Perceptive Hierarchical-Task MPC for Sequential Mobile Manipulation in Unstructured Semi-Static Environments
• Abstract summary: sequential mobile manipulation is not solely dependent on consistent motion generation but also on the robot's awareness and adaptivity to changes in the operating environment.<n>We propose a novel perceptive hierarchical-task model predictive control (HTMPC) framework for efficient sequential mobile manipulation in unstructured, changing environments.
• Score: 10.890430040930971
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As compared to typical mobile manipulation tasks, sequential mobile manipulation poses a unique challenge -- as the robot operates over extended periods, successful task completion is not solely dependent on consistent motion generation but also on the robot's awareness and adaptivity to changes in the operating environment. While existing motion planners can generate whole-body trajectories to complete sequential tasks, they typically assume that the environment remains static and rely on precomputed maps. This assumption often breaks down during long-term operations, where semi-static changes such as object removal, introduction, or shifts are common. In this work, we propose a novel perceptive hierarchical-task model predictive control (HTMPC) framework for efficient sequential mobile manipulation in unstructured, changing environments. To tackle the challenge, we leverage a Bayesian inference framework to explicitly model object-level changes and thereby maintain a temporally accurate representation of the 3D environment; this up-to-date representation is embedded in a lexicographic optimization framework to enable efficient execution of sequential tasks. We validate our perceptive HTMPC approach through both simulated and real-robot experiments. In contrast to baseline methods, our approach systematically accounts for moved and phantom obstacles, successfully completing sequential tasks with higher efficiency and reactivity, without relying on prior maps or external infrastructure.

Related papers

• Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation [17.382238044141783]
  We propose a framework for learning skills from demonstrations for robots.<n>By integrating meta-learning to construct a shared skill space, DAMI equips agents for rapid adaptation to novel tasks.<n>Our approach outperforms state-of-the-art baselines regarding direct inference on seen tasks and adaptation to unseen tasks.
  arXiv  Detail & Related papers  (2026-07-17T11:50:03Z)
• F2F-AP: Flow-to-Future Asynchronous Policy for Real-time Dynamic Manipulation [62.06267255986041]
  Asynchronous inference has emerged as a prevalent paradigm in robotic manipulation.<n>In this paper, we propose a novel framework that leverages predicted object flow to synthesize future observations.<n>Our approach significantly enhances responsiveness and success rates in complex dynamic manipulation tasks.
  arXiv  Detail & Related papers  (2026-04-02T17:57:15Z)
• UMBRELLA: Uncertainty-aware Multi-robot Reactive Coordination under Dynamic Temporal Logic Tasks [5.794342083222513]
  This paper addresses the problem of coordinating multi-robot systems for collaborative tasks involving dynamic and moving targets.<n>We explicitly model the uncertainty in target motion prediction via Conformal Prediction (CP)<n>The proposed framework (UMBRELLA) combines the Monte Carlo Tree Search (MCTS) over partial plans with uncertainty-aware rollouts.
  arXiv  Detail & Related papers  (2026-03-26T12:40:04Z)
• Hierarchical Task Model Predictive Control for Sequential Mobile Manipulation Tasks [10.635304654522814]
  We propose a novel Hierarchical-Task Model Predictive Control framework that is able to complete sequential tasks with improved performance and reactivity by effectively leveraging the robot's redundancy.<n>Compared to a typical single-task architecture, our proposed hierarchical task control architecture enables the robot to traverse a shorter path in task space and achieves an execution time 2.3 times faster when executing a sequence of delivery tasks.
  arXiv  Detail & Related papers  (2026-03-10T21:07:02Z)
• PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention [92.85371254435074]
  PosA-VLA framework anchors visual attention via pose-conditioned supervision, consistently guiding the model's perception toward task-relevant regions.<n>We show that our method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks.
  arXiv  Detail & Related papers  (2025-12-03T12:14:29Z)
• Unlocking Smarter Device Control: Foresighted Planning with a World Model-Driven Code Execution Approach [82.27842884709378]
  We propose a framework that prioritizes natural language understanding and structured reasoning to enhance the agent's global understanding of the environment.<n>Our method outperforms previous approaches, particularly achieving a 44.4% relative improvement in task success rate.
  arXiv  Detail & Related papers  (2025-05-22T09:08:47Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up.<n>It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention.<n>It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• Diffusion Transformer Policy [48.50988753948537]
  We propose a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, to model continuous end-effector actions.<n>By leveraging the scaling capability of transformers, the proposed approach can effectively model continuous end-effector actions across large diverse robot datasets.
  arXiv  Detail & Related papers  (2024-10-21T12:43:54Z)
• Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives [89.34229413345541]
  We propose a conditioning scheme which avoids pitfalls by learning the controller and its conditioning in an end-to-end manner. Our model predicts complex action sequences based directly on a dynamic image representation of the robot motion. We report significant improvements in task success over representative MPC and IL baselines.
  arXiv  Detail & Related papers  (2020-03-19T15:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.