GraspXL: Generating Grasping Motions for Diverse Objects at Scale

• URL: http://arxiv.org/abs/2403.19649v2
• Date: Fri, 12 Jul 2024 17:48:07 GMT
• Title: GraspXL: Generating Grasping Motions for Diverse Objects at Scale
• Authors: Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, Jie Song,
• Abstract summary: We unify the generation of hand-object grasping motions across multiple motion objectives in a policy learning framework GraspXL. Our policy trained with 58 objects can robustly synthesize diverse grasping motions for more than 500k unseen objects with a success rate of 82.2%. Our framework can be deployed to different dexterous hands and work with reconstructed or generated objects.
• Score: 30.104108863264706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human hands possess the dexterity to interact with diverse objects such as grasping specific parts of the objects and/or approaching them from desired directions. More importantly, humans can grasp objects of any shape without object-specific skills. Recent works synthesize grasping motions following single objectives such as a desired approach heading direction or a grasping area. Moreover, they usually rely on expensive 3D hand-object data during training and inference, which limits their capability to synthesize grasping motions for unseen objects at scale. In this paper, we unify the generation of hand-object grasping motions across multiple motion objectives, diverse object shapes and dexterous hand morphologies in a policy learning framework GraspXL. The objectives are composed of the graspable area, heading direction during approach, wrist rotation, and hand position. Without requiring any 3D hand-object interaction data, our policy trained with 58 objects can robustly synthesize diverse grasping motions for more than 500k unseen objects with a success rate of 82.2%. At the same time, the policy adheres to objectives, which enables the generation of diverse grasps per object. Moreover, we show that our framework can be deployed to different dexterous hands and work with reconstructed or generated objects. We quantitatively and qualitatively evaluate our method to show the efficacy of our approach. Our model, code, and the large-scale generated motions are available at https://eth-ait.github.io/graspxl/.

Related papers

• WANDR: Intention-guided Human Motion Generation [67.07028110459787]
  We introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location. Intention guides the agent to the goal, and interactively adapts the generation to novel situations without needing to define sub-goals or the entire motion path. We evaluate our method extensively and demonstrate its ability to generate natural and long-term motions that reach 3D goals and to unseen goal locations.
  arXiv  Detail & Related papers  (2024-04-23T10:20:17Z)
• Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction [8.253265795150401]
  This paper introduces the first text-guided work for generating the sequence of hand-object interaction in 3D. For contact generation, a VAE-based network takes as input a text and an object mesh, and generates the probability of contacts between the surfaces of hands and the object. For motion generation, a Transformer-based diffusion model utilizes this 3D contact map as a strong prior for generating physically plausible hand-object motion.
  arXiv  Detail & Related papers  (2024-03-31T04:56:30Z)
• Controllable Human-Object Interaction Synthesis [77.56877961681462]
  We propose Controllable Human-Object Interaction Synthesis (CHOIS) to generate synchronized object motion and human motion in 3D scenes. Here, language descriptions inform style and intent, and waypoints, which can be effectively extracted from high-level planning, ground the motion in the scene. Our module seamlessly integrates with a path planning module, enabling the generation of long-term interactions in 3D environments.
  arXiv  Detail & Related papers  (2023-12-06T21:14:20Z)
• Anthropomorphic Grasping with Neural Object Shape Completion [20.952799332420195]
  Humans exhibit extraordinary dexterity when handling objects. Hand postures commonly demonstrate the influence of specific regions on objects that need to be grasped. In this work, we leverage human-like object understanding by reconstructing and completing their full geometry from partial observations.
  arXiv  Detail & Related papers  (2023-11-04T21:05:26Z)
• Object Motion Guided Human Motion Synthesis [22.08240141115053]
  We study the problem of full-body human motion synthesis for the manipulation of large-sized objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework. We develop a novel system that captures full-body human manipulation motions by simply attaching a smartphone to the object being manipulated.
  arXiv  Detail & Related papers  (2023-09-28T08:22:00Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)
• Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies [29.998917158604694]
  We present a reinforcement learning framework that learns the interactive grasping of various geometrically distinct real-world objects. Videos of learned interactive policies are available at https://maltemosbach.org/io/geometry_aware_grasping_policies.
  arXiv  Detail & Related papers  (2022-11-20T11:47:33Z)
• D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions [47.55376158184854]
  We introduce the dynamic synthesis grasp task. Given an object with a known 6D pose and a grasp reference, our goal is to generate motions that move the object to a target 6D pose. A hierarchical approach decomposes the task into low-level grasping and high-level motion synthesis.
  arXiv  Detail & Related papers  (2021-12-01T17:04:39Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.