PRISM: Projection-Integrated Sampling-Based MPC with Bayesian Cost Tuning for Bimanual Manipulation

• URL: http://arxiv.org/abs/2608.25666v1
• Date: Wed, 26 Aug 2026 11:47:12 GMT
• Title: PRISM: Projection-Integrated Sampling-Based MPC with Bayesian Cost Tuning for Bimanual Manipulation
• Abstract summary: Bimanual manipulation in cluttered, contact-rich environments requires coordinated motion generation, interaction-aware planning, and reliable execution under tight kinematic constraints.<n>We present PRISM, a projection-integrated sampling-based Model Predictive Control framework that uses a GPU-accelerated physics simulator as an online world model for complex dual-arm manipulation.
• Score: 1.9669271611151167
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bimanual manipulation in cluttered, contact-rich environments remains challenging because it requires coordinated motion generation, interaction-aware planning, and reliable execution under tight kinematic constraints. We present PRISM, a projection-integrated sampling-based Model Predictive Control (MPC) framework that uses a GPU-accelerated physics simulator as an online world model for complex dual-arm manipulation. The main algorithmic contribution is a QP-guided control sampling strategy that decouples trajectory exploration from kinematic feasibility. At each MPC step, sampled joint-velocity trajectories are projected onto the set of motions satisfying joint position, velocity, acceleration, and jerk bounds, together with an initial-velocity boundary condition, before rollout evaluation. This enables broad yet feasible exploration of coordinated bimanual behaviors. To support efficient online execution, we derive a custom ADMM/Bregman-splitting QP solver that exploits joint-wise separability and reusable matrix factorizations. We further use Bayesian optimization to tune task-cost weights offline, reducing manual parameter selection. We evaluate PRISM on challenging variants of PerAct$^{2}$ tasks, including obstacle-constrained ball transport, tray transport, cube handover, and box lifting. Experiments show improved robustness and task success relative to representative sampling-based baselines, while maintaining real-time or near-real-time execution. We also demonstrate successful sim-to-real transfer on dual UR5e manipulators, highlighting the practical potential of physics-based online planning for contact-rich bimanual manipulation. Project details, including code and supplementary videos, are available at \href{https://sites.google.com/view/prismbimanual}{\texttt{https://sites.google.com/view/prismbimanual}}.

Related papers

• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding [58.541098216700796]
  evaluating the physical consistency of embodied world models is a critical open challenge.<n>Existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction.<n>We present KineBench, an IDM-free closed-loop benchmark for EWMs.
  arXiv  Detail & Related papers  (2026-07-22T08:04:17Z)
• CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation [61.25264898597319]
  We show that complex manipulation capabilities can emerge naturally from the composition of simple, independent behaviors.<n>We propose CoStream, a framework orchestrating foundation models and diverse sensing modalities into multiple composable core behaviors.<n>We demonstrate CoStream on 8 real-world tasks spanning everyday manipulation and precision assembly, with the strongest gains in contact-rich assembly and object transfer.
  arXiv  Detail & Related papers  (2026-06-24T22:25:19Z)
• Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement [16.29450701834732]
  We use a physical simulator as a backend rollout engine to evaluate candidate trajectories in parallel and refine nominal actions online.<n>We develop a real-to-sim module, trained purely on synthetic data, that maps a single RGB observation to simulation-compatible cloth state.<n>Real-robot experiments show higher success rates and stronger than baseline methods.
  arXiv  Detail & Related papers  (2026-06-23T13:20:12Z)
• Real-World Deployment of Massively Parallel Sampling-Based MPC for Contact-Rich Manipulation [21.501604407170273]
  We present an MPC framework that leverages JAX for large-scale parallelization and efficient computation.<n>We deploy it on a Franka Research 3 executing the Push-T manipulation task through a complete real-to-sim-to-real pipeline.<n>These findings highlight key challenges for sampling-based MPC in contact-rich manipulation-contact sensitivity, tight compute budgets, and the difficulty of obtaining informative domain-randomization signals in real time.
  arXiv  Detail & Related papers  (2026-06-16T12:24:05Z)
• Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation [3.88671418022057]
  We present CORD-SLS, a real-time control method for safe deformable object manipulation, with a focus on ropes and cloth.<n>At its core is a GPU-parallel differentiable simulator with contact smoothing which enables efficient gradient-based planning.<n>We develop a real-time, GPU-parallel output-feedback robust model predictive control (MPC) algorithm that plans with this simulator.
  arXiv  Detail & Related papers  (2026-06-12T07:10:56Z)
• ParaMaP: Parallel Mapping and Collision-free Motion Planning for Reactive Robot Manipulation [17.260700135265267]
  Real-time and collision-free motion planning remains challenging for robotic manipulation in unknown environments.<n>We propose a parallel mapping and motion planning framework that tightly integrates Euclidean Distance Transform (EDT)-based environment representation with a sampling-based model predictive control (SMPC) planner.<n>The effectiveness of the proposed framework is validated through extensive simulations and real-world experiments on a 7-DoF robotic manipulator.
  arXiv  Detail & Related papers  (2025-12-27T12:24:10Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up.<n>It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention.<n>It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Driving [27.776472262857045]
  This paper presents a Simple and effIcient Motion Prediction baseLine (SIMPL) for autonomous vehicles. We propose a compact and efficient global feature fusion module that performs directed message passing in a symmetric manner. As a strong baseline, SIMPL exhibits highly competitive performance on Argoverse 1 & 2 motion forecasting benchmarks.
  arXiv  Detail & Related papers  (2024-02-04T15:07:49Z)
• TLControl: Trajectory and Language Control for Human Motion Synthesis [68.09806223962323]
  We present TLControl, a novel method for realistic human motion synthesis. It incorporates both low-level Trajectory and high-level Language semantics controls. It is practical for interactive and high-quality animation generation.
  arXiv  Detail & Related papers  (2023-11-28T18:54:16Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.