C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video

• URL: http://arxiv.org/abs/2608.07045v1
• Date: Fri, 07 Aug 2026 09:54:00 GMT
• Title: C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video
• Abstract summary: C2Dex is a video-to-dexterous-manipulation framework built around a shared interaction.<n> stable object-side contacts recovered by noisy frame-wise observations in the canonical object space.<n>C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%.
• Score: 47.458936125974226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-quality demonstrations for dexterous robot manipulation are costly and difficult to collect, whereas monocular human videos provide a scalable source of diverse manipulation behaviors. However, transferring such demonstrations to dexterous robots remains challenging: monocular hand-object interaction (HOI) reconstruction often produces temporally unstable contacts and physically implausible interactions, while conventional retargeting methods struggle to preserve task-relevant contacts and local interaction geometry across different hand embodiments. We present C2Dex, a video-to-dexterous-manipulation framework built around a shared interaction representation: stable object-side contacts recovered by aggregating noisy frame-wise observations in the canonical object space. These stable contacts serve a dual role: as trajectory-level constraints that guide reconstruction toward temporally coherent and physically plausible human HOI trajectories, and as explicit transfer targets for the dexterous hand, where Laplacian interaction optimization preserves the local hand-object geometry across embodiments and residual reinforcement learning refines the trajectory in simulation. Experiments on DexYCB and TACO show that C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%, respectively, substantially outperforming the strongest baselines (17.78% and 10.00%) under identical evaluation criteria. Real-robot replay experiments further demonstrate physical feasibility across diverse contact-rich manipulation tasks. Project page: https://k-jie.github.io/C2Dex/

Related papers

• ProxiDex: Learning Dynamics-Guided Proximity Policy for Dexterous Manipulation [9.170854062817556]
  ProxiDex is a dynamics-guided proximity policy framework for dexterous manipulation.<n>It reconstructs interaction point clouds and converts geometric distances into proximity cues, forming a hardware-agnostic contact representation.<n>ProxiDex adaptively reweights proximity tokens across manipulation phases and uses dynamics-consistency supervision to guide policy inference.
  arXiv  Detail & Related papers  (2026-09-15T03:31:55Z)
• One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry [49.066054862620014]
  Learning from human demonstrations has emerged as a scalable alternative to robot teleoperation.<n>Recent sim-to-real RL methods incorporate such priors, but often omit rewards that explicitly incentivize precise contact.<n>We propose DemoMimic, a policy that manipulates objects by focusing on their geometry local to the contact points.
  arXiv  Detail & Related papers  (2026-09-01T23:04:35Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• Hand-centric Human-to-Robot Trajectory Transfer from Video Demonstrations via Open-World Contact Localization [28.23926683554352]
  We present emphHOWTransfer, a hand-centric framework that distills human demonstrations into contact-aware, taxonomy-informed, and diverse robotic trajectories.<n>emphHOWTransfer recovers temporally consistent 3D hand motion and localizes temporal contact intervals by reasoning over observed hand-object interaction cues.<n>Experiments show that emphHOWTransfer enables accurate contact localization and high-quality robot motion with $86%$ success.
  arXiv  Detail & Related papers  (2026-06-09T11:53:29Z)
• SECOND-Grasp: Semantic Contact-guided Dexterous Grasping [60.1519218638742]
  Second-Grasp (SEmantic CONtact-guided Dexterous Grasping) is a unified framework that enables robotic hands to adjust grasping strategies based on semantic reasoning.<n>We show that our approach consistently outperforms baselines in lifting success rate on both seen and unseen categories.
  arXiv  Detail & Related papers  (2026-05-13T07:37:00Z)
• SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance [49.69016078147708]
  Accurately reconstructing human behavior in close-interaction scenarios is crucial for enabling realistic virtual interactions in augmented reality.<n>We propose SocialMirror, a diffusion-based framework that integrates semantic and geometric cues to effectively address these issues.<n>SocialMirror achieves state-of-the-art performance in reconstructing interactive human meshes, demonstrating strong generalization across unseen datasets and in-the-wild scenarios.
  arXiv  Detail & Related papers  (2026-04-15T07:41:52Z)
• FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation [9.630733148732475]
  We propose FAR-Dex, a hierarchical framework that integrates few-shot data augmentation with adaptive residual refinement to enable robust and precise arm-hand coordination in dexterous tasks.<n>Experiments in both simulation and real-world demonstrate that FAR-Dex improves data quality by 13.4% and task success rates by 7% over state-of-the-art methods.<n>It further achieves over 80% success in real-world tasks, enabling fine-grained dexterous manipulation with strong positional generalization.
  arXiv  Detail & Related papers  (2026-03-11T06:10:03Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation [45.753757870577196]
  We introduce AGILE, a robust framework that shifts the paradigm from reconstruction to agentic generation for interaction learning.<n>We show that AGILE outperforms baselines in global geometric accuracy while demonstrating exceptional robustness on challenging sequences where prior art frequently collapses.
  arXiv  Detail & Related papers  (2026-02-04T15:42:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.