BodySLAM: Joint Camera Localisation, Mapping, and Human Motion Tracking
- URL: http://arxiv.org/abs/2205.02301v1
- Date: Wed, 4 May 2022 19:38:26 GMT
- Title: BodySLAM: Joint Camera Localisation, Mapping, and Human Motion Tracking
- Authors: Dorian Henning, Tristan Laidlow, Stefan Leutenegger
- Abstract summary: Estimating human motion from video is an active research area due to its many potential applications.
We present BodySLAM, a monocular SLAM system that jointly estimates the position, shape, and posture of human bodies.
We also introduce a novel human motion model to constrain sequential body postures and observe the scale of the scene.
- Score: 19.123370976371277
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Estimating human motion from video is an active research area due to its many
potential applications. Most state-of-the-art methods predict human shape and
posture estimates for individual images and do not leverage the temporal
information available in video. Many "in the wild" sequences of human motion
are captured by a moving camera, which adds the complication of conflated
camera and human motion to the estimation. We therefore present BodySLAM, a
monocular SLAM system that jointly estimates the position, shape, and posture
of human bodies, as well as the camera trajectory. We also introduce a novel
human motion model to constrain sequential body postures and observe the scale
of the scene. Through a series of experiments on video sequences of human
motion captured by a moving monocular camera, we demonstrate that BodySLAM
improves estimates of all human body parameters and camera poses when compared
to estimating these separately.
Related papers
- WHAC: World-grounded Humans and Cameras [37.877565981937586]
We aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly.
We introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation.
We present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras.
arXiv Detail & Related papers (2024-03-19T17:58:02Z) - PACE: Human and Camera Motion Estimation from in-the-wild Videos [113.76041632912577]
We present a method to estimate human motion in a global scene from moving cameras.
This is a highly challenging task due to the coupling of human and camera motions in the video.
We propose a joint optimization framework that disentangles human and camera motions using both foreground human motion priors and background scene features.
arXiv Detail & Related papers (2023-10-20T19:04:14Z) - Decoupling Human and Camera Motion from Videos in the Wild [67.39432972193929]
We propose a method to reconstruct global human trajectories from videos in the wild.
Our method decouples the camera and human motion, which allows us to place people in the same world coordinate frame.
arXiv Detail & Related papers (2023-02-24T18:59:15Z) - D&D: Learning Human Dynamics from Dynamic Camera [55.60512353465175]
We present D&D (Learning Human Dynamics from Dynamic Camera), which leverages the laws of physics to reconstruct 3D human motion from the in-the-wild videos with a moving camera.
Our approach is entirely neural-based and runs without offline optimization or simulation in physics engines.
arXiv Detail & Related papers (2022-09-19T06:51:02Z) - Embodied Scene-aware Human Pose Estimation [25.094152307452]
We propose embodied scene-aware human pose estimation.
Our method is one stage, causal, and recovers global 3D human poses in a simulated environment.
arXiv Detail & Related papers (2022-06-18T03:50:19Z) - GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras [99.07219478953982]
We present an approach for 3D global human mesh recovery from monocular videos recorded with dynamic cameras.
We first propose a deep generative motion infiller, which autoregressively infills the body motions of occluded humans based on visible motions.
In contrast to prior work, our approach reconstructs human meshes in consistent global coordinates even with dynamic cameras.
arXiv Detail & Related papers (2021-12-02T18:59:54Z) - Human Performance Capture from Monocular Video in the Wild [50.34917313325813]
We propose a method capable of capturing the dynamic 3D human shape from a monocular video featuring challenging body poses.
Our method outperforms state-of-the-art methods on an in-the-wild human video dataset 3DPW.
arXiv Detail & Related papers (2021-11-29T16:32:41Z) - Human POSEitioning System (HPS): 3D Human Pose Estimation and
Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment.
We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift.
HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
arXiv Detail & Related papers (2021-03-31T17:58:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.