Learned Neural Physics Simulation for Articulated 3D Human Pose Reconstruction

• URL: http://arxiv.org/abs/2410.12023v1
• Date: Tue, 15 Oct 2024 19:42:45 GMT
• Title: Learned Neural Physics Simulation for Articulated 3D Human Pose Reconstruction
• Authors: Mykhaylo Andriluka, Baruch Tabanpour, C. Daniel Freeman, Cristian Sminchisescu,
• Abstract summary: We propose a novel neural network approach, LARP, to model the dynamics of articulated human motion with contact. Our neural architecture supports features typically found in traditional physics simulators. To demonstrate the value of LARP we use it as a drop-in replacement for a state of the art classical non-differentiable simulator in an existing video-based reconstruction framework.
• Score: 30.51621591645056
• License:
• Abstract: We propose a novel neural network approach, LARP (Learned Articulated Rigid body Physics), to model the dynamics of articulated human motion with contact. Our goal is to develop a faster and more convenient methodological alternative to traditional physics simulators for use in computer vision tasks such as human motion reconstruction from video. To that end we introduce a training procedure and model components that support the construction of a recurrent neural architecture to accurately simulate articulated rigid body dynamics. Our neural architecture supports features typically found in traditional physics simulators, such as modeling of joint motors, variable dimensions of body parts, contact between body parts and objects, and is an order of magnitude faster than traditional systems when multiple simulations are run in parallel. To demonstrate the value of LARP we use it as a drop-in replacement for a state of the art classical non-differentiable simulator in an existing video-based reconstruction framework and show comparative or better 3D human pose reconstruction accuracy.

Related papers

• Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics [48.99021224773799]
  We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections. We also propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images.
  arXiv  Detail & Related papers  (2024-10-10T17:43:36Z)
• PhyRecon: Physically Plausible Neural Scene Reconstruction [81.73129450090684]
  We introduce PHYRECON, the first approach to leverage both differentiable rendering and differentiable physics simulation to learn implicit surface representations. Central to this design is an efficient transformation between SDF-based implicit representations and explicit surface points. Our results also exhibit superior physical stability in physical simulators, with at least a 40% improvement across all datasets.
  arXiv  Detail & Related papers  (2024-04-25T15:06:58Z)
• A Physics-embedded Deep Learning Framework for Cloth Simulation [6.8806198396336935]
  This paper proposes a physics-embedded learning framework that directly encodes physical features of cloth simulation. The framework can also integrate with other external forces and collision handling through either traditional simulators or sub neural networks.
  arXiv  Detail & Related papers  (2024-03-19T15:21:00Z)
• NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos [82.74918564737591]
  We present a method for learning 3D geometry and physics parameters of a dynamic scene from only a monocular RGB video input. Experiments show that our method achieves superior mesh and video reconstruction of dynamic scenes compared to competing Neural Field approaches.
  arXiv  Detail & Related papers  (2022-10-22T04:57:55Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)
• Trajectory Optimization for Physics-Based Reconstruction of 3d Human Pose from Monocular Video [31.96672354594643]
  We focus on the task of estimating a physically plausible articulated human motion from monocular video. Existing approaches that do not consider physics often produce temporally inconsistent output with motion artifacts. We show that our approach achieves competitive results with respect to existing physics-based methods on the Human3.6M benchmark.
  arXiv  Detail & Related papers  (2022-05-24T18:02:49Z)
• Differentiable Dynamics for Articulated 3d Human Motion Reconstruction [29.683633237503116]
  We introduce DiffPhy, a differentiable physics-based model for articulated 3d human motion reconstruction from video. We validate the model by demonstrating that it can accurately reconstruct physically plausible 3d human motion from monocular video.
  arXiv  Detail & Related papers  (2022-05-24T17:58:37Z)
• Inferring Articulated Rigid Body Dynamics from RGBD Video [18.154013621342266]
  We introduce a pipeline that combines inverse rendering with differentiable simulation to create digital twins of real-world articulated mechanisms. Our approach accurately reconstructs the kinematic tree of an articulated mechanism being manipulated by a robot.
  arXiv  Detail & Related papers  (2022-03-20T08:19:02Z)
• H4D: Human 4D Modeling by Learning Neural Compositional Representation [75.34798886466311]
  This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation. Experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks.
  arXiv  Detail & Related papers  (2022-03-02T17:10:49Z)
• Learning Local Recurrent Models for Human Mesh Recovery [50.85467243778406]
  We present a new method for video mesh recovery that divides the human mesh into several local parts following the standard skeletal model. We then model the dynamics of each local part with separate recurrent models, with each model conditioned appropriately based on the known kinematic structure of the human body. This results in a structure-informed local recurrent learning architecture that can be trained in an end-to-end fashion with available annotations.
  arXiv  Detail & Related papers  (2021-07-27T14:30:33Z)
• Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation [25.03715978502528]
  We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation. We use a kinematics model of the human body to represent the entire range of human motion, and a dynamics model of the body to interact with objects inside a physics simulator. This is the first work to estimate a physically valid 3D full-body interaction sequence with objects from egocentric videos.
  arXiv  Detail & Related papers  (2020-11-10T00:06:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.