Grasp as You Dream: Imitating Functional Grasping from Generated Human Demonstrations

• URL: http://arxiv.org/abs/2604.07517v1
• Date: Wed, 08 Apr 2026 18:52:16 GMT
• Title: Grasp as You Dream: Imitating Functional Grasping from Generated Human Demonstrations
• Abstract summary: We present GraspDreamer, a method to enable zero-shot functional grasping without labor-intensive data collection.<n>The key idea is that VGMs pre-trained on internet-scale human data implicitly encode generalized priors about how humans interact with the physical world.<n>Extensive experiments on the public benchmarks with different robot hands demonstrate the superior data efficiency and generalization performance of GraspDreamer.
• Score: 23.294838162593184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Building generalist robots capable of performing functional grasping in everyday, open-world environments remains a significant challenge due to the vast diversity of objects and tasks. Existing methods are either constrained to narrow object/task sets or rely on prohibitively large-scale data collection to capture real-world variability. In this work, we present an alternative approach, GraspDreamer, a method that leverages human demonstrations synthesized by visual generative models (VGMs) (e.g., video generation models) to enable zero-shot functional grasping without labor-intensive data collection. The key idea is that VGMs pre-trained on internet-scale human data implicitly encode generalized priors about how humans interact with the physical world, which can be combined with embodiment-specific action optimization to enable functional grasping with minimal effort. Extensive experiments on the public benchmarks with different robot hands demonstrate the superior data efficiency and generalization performance of GraspDreamer compared to previous methods. Real-world evaluations further validate the effectiveness on real robots. Additionally, we showcase that GraspDreamer can (1) be naturally extended to downstream manipulation tasks, and (2) can generate data to support visuomotor policy learning.

Related papers

• HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning [52.022681285103126]
  We present HumanoidMimicGen, a method for generating humanoid legged loco-manipulation data.<n>Our method adapts contact-rich whole-body skills from a handful of source demonstrations to new states.<n>We show that whole-body visuomotor policies co-trained with data generated by HumanoidMimicGen outperform those trained only on real-world data by 20%.
  arXiv  Detail & Related papers  (2026-05-26T21:57:11Z)
• BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  BifrostUMI is a portable, efficient, and robot-free data collection framework for humanoid robots.<n>We demonstrate the efficacy and versatility of the proposed framework across two distinct experimental scenarios.
  arXiv  Detail & Related papers  (2026-05-05T07:35:09Z)
• GazeVLA: Learning Human Intention for Robotic Manipulation [22.106797717672293]
  Embodied foundation models have achieved significant breakthroughs in robotic manipulation, yet they still depend heavily on large-scale robot demonstrations.<n>We argue that the intention underlying human actions can serve as a powerful intermediate representation for bridging this gap.<n>We introduce a novel framework that explicitly learns and transfers human intention to facilitate robotic manipulation.
  arXiv  Detail & Related papers  (2026-04-24T14:46:03Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Emergence of Human to Robot Transfer in Vision-Language-Action Models [88.76648919814771]
  Vision-language-action (VLA) models can enable broad open world generalization, but require large and diverse datasets.<n>We show that human-to-robot transfer emerges once the VLA is pre-trained on sufficient scenes, tasks, and embodiments.
  arXiv  Detail & Related papers  (2025-12-27T00:13:11Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model [36.82365894983052]
  A major bottleneck lies in the scarcity of large-scale, action-annotated data for dexterous skills.<n>We propose METIS, a vision-language-action model for dexterous manipulation pretrained on egocentric datasets.<n>Our method demonstrates exceptional dexterous manipulation capabilities, achieving highest average success rate in six real-world tasks.
  arXiv  Detail & Related papers  (2025-11-21T16:32:36Z)
• Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation [16.809190349155525]
  We propose a novel adaptation paradigm that leverages readily available paired human-robot video data to bridge the domain gap.<n>Our method employs a human-robot contrastive alignment loss to align the semantics of human and robot videos, adapting pre-trained models to the robot domain in a parameter-efficient manner.
  arXiv  Detail & Related papers  (2024-06-20T11:57:46Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.<n>Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.<n>Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.