Grasp, Handover, Rotate: Bimanual Object Reorientation via Compositional Diffusion and Energy-Based Optimization

• URL: http://arxiv.org/abs/2607.21341v1
• Date: Thu, 23 Jul 2026 14:11:14 GMT
• Title: Grasp, Handover, Rotate: Bimanual Object Reorientation via Compositional Diffusion and Energy-Based Optimization
• Abstract summary: BiCompoDiff is a compositional diffusion and energy-based framework that jointly optimize grasp selection, handover, regrasp, and motion planning.<n> Experiments across diverse simulated household reorientation tasks demonstrate that BiCompoDiff achieves over 20% higher success rates.<n>Real-world validation confirms effective sim-to-real transfer and robust performance on challenging scenes.
• Score: 28.020553566509523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bimanual object reorientation - picking an object, handing it over between two arms, and placing it in a desired target pose - is valuable when direct placement from the initial grasp is infeasible due to collisions, kinematic constraints, or poor final orientation. However, achieving this under multiple competing objectives remains challenging. We introduce BiCompoDiff, a compositional diffusion and energy-based framework that jointly optimizes grasp selection, handover, regrasp, and motion planning under multiple constraints. By combining a pretrained grasp diffusion model with bimanual planning energy-based models (EBMs), our method injects gradient guidance during reverse diffusion to enforce collision avoidance, trajectory smoothness (via differentiable inverse kinematics), handover feasibility, and regrasp safety. Annealed MCMC sampling further refines grasp poses over the composite energy landscape. Experiments across diverse simulated household reorientation tasks demonstrate that BiCompoDiff achieves over 20% higher success rates and up to 37% smoother trajectories (measured by joint displacement) compared to strong sampling-based baselines. Real-world validation confirms effective sim-to-real transfer and robust performance on challenging scenes.

Related papers

• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• Motion Planning with Model-Based Diffusion via Constraint Optimization and Adaptive Scheduling [7.761673975696802]
  Single-Robot Motion Planning (SRMP) is challenging under complex constraints and computational limits.<n>Recent Model-Based Diffusion (MBD) recasts the SRMP to guide samples from a posterior diffusion trajectories.<n>Our method achieves higher safety & success rates, faster convergence, and lower final costs than baseline planners.
  arXiv  Detail & Related papers  (2026-07-16T01:04:32Z)
• D$^3$-MoE:Dual Disentangled Diffusion Mixture-of-Experts for Style-Controllable End-to-End Autonomous Driving [22.70855312181222]
  D$3$-MoE disentangles trajectory modeling along two complementary axes.<n>On the behavioral axis, generation is decoupled from selection.<n>On the physical axis, decoupled longitudinal and lateral routers activate their respective experts.
  arXiv  Detail & Related papers  (2026-06-03T13:46:46Z)
• AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models [60.04879435087352]
  Vision-language-action (VLA) policies generate actions within a single unified space.<n>We propose AnchorRefine, a hierarchical framework that factorizes VLA action modeling into trajectory anchor and residual refinement.<n> Experiments on LIBERO, CALVIN, and real-robot tasks demonstrate that AnchorRefine consistently improves both regression-based and diffusion-based VLA backbones.
  arXiv  Detail & Related papers  (2026-04-20T04:25:24Z)
• Differentiable Object Pose Connectivity Metrics for Regrasp Sequence Optimization [13.078429762098631]
  Regrasp planning is often required when one pick-and-place cannot transfer an object from an initial pose to a goal pose.<n>We propose an implicit multi-step regrasp planning framework based on differentiable pose sequence connectivity metrics.
  arXiv  Detail & Related papers  (2026-04-16T07:47:57Z)
• Hypergraph-State Collaborative Reasoning for Multi-Object Tracking [63.32950991964095]
  We propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects.<n>By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded.
  arXiv  Detail & Related papers  (2026-04-14T12:37:39Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• DRIFT: Diffusion-based Rule-Inferred For Trajectories [15.78457153766977]
  Trajectory generation for mobile robots in unstructured environments faces a critical dilemma: balancing smoothness for safe execution with terminal precision for fine-grained tasks.<n>This article proposes DRIFT, a conditional framework designed to generate high-fidelity reference trajectories by integrating two complementary inductive biases.<n>Results demonstrate that DRIFT reconciles these conflicting objectives, achieving centimeter-level imitation fidelity (0.041m FDE) and competitive smoothness (27.19 Jerk)
  arXiv  Detail & Related papers  (2026-03-01T05:54:38Z)
• Learning to Align and Refine: A Foundation-to-Diffusion Framework for Occlusion-Robust Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a dual-stage Foundation-to-Diffusion framework that precisely align 2D prior guidance from vision foundation models.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>To resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic vectors into the local matching process.<n>The method improves the $Survival_50$ metric by 17.9% over event-only tracking of any point baseline.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• Deep Semantic Matching with Foreground Detection and Cycle-Consistency [103.22976097225457]
  We address weakly supervised semantic matching based on a deep network. We explicitly estimate the foreground regions to suppress the effect of background clutter. We develop cycle-consistent losses to enforce the predicted transformations across multiple images to be geometrically plausible and consistent.
  arXiv  Detail & Related papers  (2020-03-31T22:38:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.