WANDR: Intention-guided Human Motion Generation

• URL: http://arxiv.org/abs/2404.15383v1
• Date: Tue, 23 Apr 2024 10:20:17 GMT
• Title: WANDR: Intention-guided Human Motion Generation
• Authors: Markos Diomataris, Nikos Athanasiou, Omid Taheri, Xi Wang, Otmar Hilliges, Michael J. Black,
• Abstract summary: We introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location. Intention guides the agent to the goal, and interactively adapts the generation to novel situations without needing to define sub-goals or the entire motion path. We evaluate our method extensively and demonstrate its ability to generate natural and long-term motions that reach 3D goals and to unseen goal locations.
• Score: 67.07028110459787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthesizing natural human motions that enable a 3D human avatar to walk and reach for arbitrary goals in 3D space remains an unsolved problem with many applications. Existing methods (data-driven or using reinforcement learning) are limited in terms of generalization and motion naturalness. A primary obstacle is the scarcity of training data that combines locomotion with goal reaching. To address this, we introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location. To solve this, we introduce novel intention features that drive rich goal-oriented movement. Intention guides the agent to the goal, and interactively adapts the generation to novel situations without needing to define sub-goals or the entire motion path. Crucially, intention allows training on datasets that have goal-oriented motions as well as those that do not. WANDR is a conditional Variational Auto-Encoder (c-VAE), which we train using the AMASS and CIRCLE datasets. We evaluate our method extensively and demonstrate its ability to generate natural and long-term motions that reach 3D goals and generalize to unseen goal locations. Our models and code are available for research purposes at wandr.is.tue.mpg.de.

Related papers

• AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation [60.5897687447003]
  AvatarGO is a novel framework designed to generate realistic 4D HOI scenes from textual inputs. Our framework not only generates coherent compositional motions, but also exhibits greater robustness in handling issues. As the first attempt to synthesize 4D avatars with object interactions, we hope AvatarGO could open new doors for human-centric 4D content creation.
  arXiv  Detail & Related papers  (2024-10-09T17:58:56Z)
• Universal Humanoid Motion Representations for Physics-Based Control [71.46142106079292]
  We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. We first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator.
  arXiv  Detail & Related papers  (2023-10-06T20:48:43Z)
• Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models [71.64318025625833]
  This paper presents a novel approach to generating the 3D motion of a human interacting with a target object. Our framework first generates a set of milestones and then synthesizes the motion along them. The experiments on the NSM, COUCH, and SAMP datasets show that our approach outperforms previous methods by a large margin in both quality and diversity.
  arXiv  Detail & Related papers  (2023-10-03T17:50:23Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis [21.650091018774972]
  We create a neural interaction field attached to a specific object, which outputs the distance to the valid interaction manifold given a human pose as input. This interaction field guides the sampling of an object-conditioned human motion diffusion model. We synthesize realistic motions for sitting and lifting with several objects, outperforming alternative approaches in terms of motion quality and successful action completion.
  arXiv  Detail & Related papers  (2023-07-14T17:59:38Z)
• Synthesizing Diverse Human Motions in 3D Indoor Scenes [16.948649870341782]
  We present a novel method for populating 3D indoor scenes with virtual humans that can navigate in the environment and interact with objects in a realistic manner. Existing approaches rely on training sequences that contain captured human motions and the 3D scenes they interact with. We propose a reinforcement learning-based approach that enables virtual humans to navigate in 3D scenes and interact with objects realistically and autonomously.
  arXiv  Detail & Related papers  (2023-05-21T09:22:24Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction [24.72947291987545]
  Key challenge for an agent learning to interact with the world is to reason about physical properties of objects. We propose a novel approach for modeling the dynamics of a robot's interactions directly from unlabeled 3D point clouds and images.
  arXiv  Detail & Related papers  (2020-08-02T11:04:49Z)
• Action2Motion: Conditioned Generation of 3D Human Motions [28.031644518303075]
  We aim to generateplausible human motion sequences in 3D. Each sampled sequence faithfully resembles anaturalhuman bodyarticulation dynamics. A new 3D human motion dataset, HumanAct12, is also constructed.
  arXiv  Detail & Related papers  (2020-07-30T05:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.