CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements

• URL: http://arxiv.org/abs/2609.07498v1
• Date: Mon, 07 Sep 2026 13:47:22 GMT
• Title: CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements
• Abstract summary: Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
• Score: 63.40365657068892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning. However, existing methods are largely confined to simple, planar tasks and fail to handle complex spatial movements (e.g., intricate trajectories involving rotations or flips) that are essential for robot manipulation. Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions. To this end, we introduce a scalable acquisition pipeline to collect hand-gripper paired demonstrations, governed by a rigorous protocol that prioritizes motion complexity and leverages a handheld gripper for seamless action mimicry. This yields a large-scale paired dataset comprising 6,189 episodes across 1,254 unique objects, exhibiting significantly higher spatial complexity than existing benchmarks. However, learning such complex mappings remains challenging. We observe that naive end-to-end generation of full gripper pose sequences is insufficient, as minor trajectory deviations compound rapidly under intricate dynamics. To address this, we propose a two-stage framework: Stage I predicts sparse gripper keyframes (initial and terminal) to simplify the mapping objective, while Stage II generates the full continuous action sequence conditioned on these keyframes. Furthermore, to mitigate cumulative drift, we keep the gripper's orientation being learned while post-optimizing its translation based on the grasping heuristic and kinematic consistency. In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations, significantly outperforming traditional baselines. Project page: https://cosmoh2g.github.io.

Related papers

• Fast Coordinated Bimanual Motion Planning With Hard Constraints [0.0]
  Bimanual manipulation enables complex tasks but introduces added complexity from the high number of degrees of freedom involved.<n>We propose a fast bimanual motion planning pipeline that enforces a hard transformation constraint continuously along the entire path.<n>We extensively evaluate the method in simulation across diverse environments, constraints and bimanual platforms, achieving 19.4x faster planning than prior work.
  arXiv  Detail & Related papers  (2026-08-21T10:08:47Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry [61.24189040578178]
  We propose a fully learning-based approach that directly infers moving objects from latent feature representations via attention mechanisms.<n>Our key insight is to bypass explicit correspondence estimation and instead let the model learn to implicitly disentangle object and camera motion.<n>Our approach achieves state-of-the-art motion segmentation performance with high efficiency.
  arXiv  Detail & Related papers  (2026-02-25T11:36:33Z)
• Learning Generalizable Hand-Object Tracking from Synthetic Demonstrations [71.15185260689928]
  We present a system for learning generalizable hand-object tracking controllers purely from synthetic data, without requiring any human demonstrations.<n>Our approach makes two key contributions: (1) HOP, a Hand-Object Planner, which can synthesize diverse hand-object trajectories; and (2) HOT, a Hand-Object Tracker, that bridges synthetic-to-physical transfer through reinforcement learning and interaction imitation learning.
  arXiv  Detail & Related papers  (2025-12-22T17:08:54Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learning [5.247557449370603]
  ProMoGen is a novel framework that integrates trajectory guidance with sparse anchor motion control.<n>ProMoGen supports both dual and single control paradigms within a unified training process.<n>Our approach seamlessly integrates personalized motion with structured guidance, significantly outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-23T13:51:42Z)
• Real-time Controllable Motion Transition for Characters [14.88407656218885]
  Real-time in-between motion generation is universally required in games and highly desirable in existing animation pipelines. Our approach consists of two key components: motion manifold and conditional transitioning. We show that our method is able to generate high-quality motions measured under multiple metrics.
  arXiv  Detail & Related papers  (2022-05-05T10:02:54Z)
• First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion [73.37945453998134]
  Traditional approaches to quadruped control employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In this work, these challenges are addressed by framing quadruped control as optimisation in a structured latent space. A deep generative model captures a statistical representation of feasible joint configurations, whilst complex dynamic and terminal constraints are expressed via high-level, semantic indicators. We validate the feasibility of locomotion trajectories optimised using our approach both in simulation and on a real-worldmal quadruped.
  arXiv  Detail & Related papers  (2020-07-03T07:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.