Human Motion Prediction, Reconstruction, and Generation

• URL: http://arxiv.org/abs/2502.15956v1
• Date: Fri, 21 Feb 2025 21:38:09 GMT
• Title: Human Motion Prediction, Reconstruction, and Generation
• Authors: Canxuan Gang, Yiran Wang,
• Abstract summary: This report reviews recent advancements in human motion prediction, reconstruction, and generation.<n>Human motion prediction focuses on forecasting future poses and movements from historical data.<n>Reconstruction aims to recover accurate 3D human body movements from visual inputs.<n>Motion generation synthesizes realistic and diverse motions from action labels, textual descriptions, or environmental constraints.
• Score: 1.0988643683092072
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This report reviews recent advancements in human motion prediction, reconstruction, and generation. Human motion prediction focuses on forecasting future poses and movements from historical data, addressing challenges like nonlinear dynamics, occlusions, and motion style variations. Reconstruction aims to recover accurate 3D human body movements from visual inputs, often leveraging transformer-based architectures, diffusion models, and physical consistency losses to handle noise and complex poses. Motion generation synthesizes realistic and diverse motions from action labels, textual descriptions, or environmental constraints, with applications in robotics, gaming, and virtual avatars. Additionally, text-to-motion generation and human-object interaction modeling have gained attention, enabling fine-grained and context-aware motion synthesis for augmented reality and robotics. This review highlights key methodologies, datasets, challenges, and future research directions driving progress in these fields.

Related papers

• Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis [80.50342609047091]
  Scene-aware text-to-human synthesis generates diverse indoor motion samples from the same textual description. We propose a cascaded generation strategy that factorizes text-driven scene-specific human motion generation into three stages. We jointly improve realistic human motion synthesis and robust human motion analysis in 3D scenes.
  arXiv  Detail & Related papers  (2025-03-01T06:56:58Z)
• Motion Generation Review: Exploring Deep Learning for Lifelike Animation with Manifold [4.853986914715961]
  Human motion generation involves creating natural sequences of human body poses, widely used in gaming, virtual reality, and human-computer interaction.<n>Previous work has focused on motion generation based on signals like movement, music, text, or scene background.<n>Mandela learning offers a solution by reducing data dimensionality and capturing subspaces of effective motion.
  arXiv  Detail & Related papers  (2024-12-12T08:27:15Z)
• Generating Human Motion in 3D Scenes from Text Descriptions [60.04976442328767]
  This paper focuses on the task of generating human motions in 3D indoor scenes given text descriptions of the human-scene interactions. We propose a new approach that decomposes the complex problem into two more manageable sub-problems. For language grounding of the target object, we leverage the power of large language models; for motion generation, we design an object-centric scene representation.
  arXiv  Detail & Related papers  (2024-05-13T14:30:12Z)
• LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment [27.38638713080283]
  We introduce LaserHuman, a pioneering dataset engineered to revolutionize Scene-Text-to-Motion research. LaserHuman stands out with its inclusion of genuine human motions within 3D environments. We propose a multi-conditional diffusion model, which is simple but effective, achieving state-of-the-art performance on existing datasets.
  arXiv  Detail & Related papers  (2024-03-20T05:11:10Z)
• DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics [21.00283279991885]
  We introduce DROP, a novel framework for modeling Dynamics Responses of humans using generative mOtion prior and Projective dynamics. We conduct extensive evaluations of our model across different motion tasks and various physical perturbations, demonstrating the scalability and diversity of responses.
  arXiv  Detail & Related papers  (2023-09-24T20:25:59Z)
• Human Motion Generation: A Survey [67.38982546213371]
  Human motion generation aims to generate natural human pose sequences and shows immense potential for real-world applications. Most research within this field focuses on generating human motions based on conditional signals, such as text, audio, and scene contexts. We present a comprehensive literature review of human motion generation, which is the first of its kind in this field.
  arXiv  Detail & Related papers  (2023-07-20T14:15:20Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network [9.247294820004143]
  We propose a transformer-based generative model for forecasting multiple diverse human motions. Our model first predicts the pose of the body relative to the hip joint. Then the textitHip Prediction Module predicts the trajectory of the hip movement for each predicted pose frame. We show that our system outperforms the state-of-the-art in human motion prediction while it can predict diverse multi-motion future trajectories with hip movements.
  arXiv  Detail & Related papers  (2022-09-13T23:22:33Z)
• GIMO: Gaze-Informed Human Motion Prediction in Context [75.52839760700833]
  We propose a large-scale human motion dataset that delivers high-quality body pose sequences, scene scans, and ego-centric views with eye gaze. Our data collection is not tied to specific scenes, which further boosts the motion dynamics observed from our subjects. To realize the full potential of gaze, we propose a novel network architecture that enables bidirectional communication between the gaze and motion branches.
  arXiv  Detail & Related papers  (2022-04-20T13:17:39Z)
• Scene-aware Generative Network for Human Motion Synthesis [125.21079898942347]
  We propose a new framework, with the interaction between the scene and the human motion taken into account. Considering the uncertainty of human motion, we formulate this task as a generative task. We derive a GAN based learning approach, with discriminators to enforce the compatibility between the human motion and the contextual scene.
  arXiv  Detail & Related papers  (2021-05-31T09:05:50Z)
• Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes [27.443701512923177]
  We propose to bridge human motion synthesis and scene affordance reasoning. We present a hierarchical generative framework to synthesize long-term 3D human motion conditioning on the 3D scene structure. Our experiments show significant improvements over previous approaches on generating natural and physically plausible human motion in a scene.
  arXiv  Detail & Related papers  (2020-12-10T09:09:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.