On the role of depth predictions for 3D human pose estimation

• URL: http://arxiv.org/abs/2103.02521v1
• Date: Wed, 3 Mar 2021 16:51:38 GMT
• Title: On the role of depth predictions for 3D human pose estimation
• Authors: Alec Diaz-Arias, Mitchell Messmore, Dmitriy Shin, and Stephen Baek
• Abstract summary: We build a system that takes 2d joint locations as input along with their estimated depth value and predicts their 3d positions in camera coordinates. Results are produced on neural network that accepts a low dimensional input and be integrated into a real-time system. Our system can be combined with an off-the-shelf 2d pose detector and a depth map predictor to perform 3d pose estimation in the wild.
• Score: 0.04199844472131921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Following the successful application of deep convolutional neural networks to 2d human pose estimation, the next logical problem to solve is 3d human pose estimation from monocular images. While previous solutions have shown some success, they do not fully utilize the depth information from the 2d inputs. With the goal of addressing this depth ambiguity, we build a system that takes 2d joint locations as input along with their estimated depth value and predicts their 3d positions in camera coordinates. Given the inherent noise and inaccuracy from estimating depth maps from monocular images, we perform an extensive statistical analysis showing that given this noise there is still a statistically significant correlation between the predicted depth values and the third coordinate of camera coordinates. We further explain how the state-of-the-art results we achieve on the H3.6M validation set are due to the additional input of depth. Notably, our results are produced on neural network that accepts a low dimensional input and be integrated into a real-time system. Furthermore, our system can be combined with an off-the-shelf 2d pose detector and a depth map predictor to perform 3d pose estimation in the wild.

Related papers

• SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning [17.99904937160487]
  We introduce SCIPaD, a novel approach that incorporates spatial clues for unsupervised depth-pose joint learning. SCIPaD achieves a reduction of 22.2% in average translation error and 34.8% in average angular error for camera pose estimation task on the KITTI Odometry dataset.
  arXiv  Detail & Related papers  (2024-07-07T06:52:51Z)
• Toward Accurate Camera-based 3D Object Detection via Cascade Depth Estimation and Calibration [20.82054596017465]
  Recent camera-based 3D object detection is limited by the precision of transforming from image to 3D feature spaces. This paper aims to address such a fundamental problem of camera-based 3D object detection: How to effectively learn depth information for accurate feature lifting and object localization.
  arXiv  Detail & Related papers  (2024-02-07T14:21:26Z)
• Scene-aware Egocentric 3D Human Pose Estimation [72.57527706631964]
  Egocentric 3D human pose estimation with a single head-mounted fisheye camera has recently attracted attention due to its numerous applications in virtual and augmented reality. Existing methods still struggle in challenging poses where the human body is highly occluded or is closely interacting with the scene. We propose a scene-aware egocentric pose estimation method that guides the prediction of the egocentric pose with scene constraints.
  arXiv  Detail & Related papers  (2022-12-20T21:35:39Z)
• Monocular 3D Object Detection with Depth from Motion [74.29588921594853]
  We take advantage of camera ego-motion for accurate object depth estimation and detection. Our framework, named Depth from Motion (DfM), then uses the established geometry to lift 2D image features to the 3D space and detects 3D objects thereon. Our framework outperforms state-of-the-art methods by a large margin on the KITTI benchmark.
  arXiv  Detail & Related papers  (2022-07-26T15:48:46Z)
• Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation from Depth Maps [66.24554680709417]
  Knowing the exact 3D location of workers and robots in a collaborative environment enables several real applications. We propose a non-invasive framework based on depth devices and deep neural networks to estimate the 3D pose of robots from an external camera.
  arXiv  Detail & Related papers  (2022-07-06T08:52:12Z)
• VR3Dense: Voxel Representation Learning for 3D Object Detection and Monocular Dense Depth Reconstruction [0.951828574518325]
  We introduce a method for jointly training 3D object detection and monocular dense depth reconstruction neural networks. It takes as inputs, a LiDAR point-cloud, and a single RGB image during inference and produces object pose predictions as well as a densely reconstructed depth map. While our object detection is trained in a supervised manner, the depth prediction network is trained with both self-supervised and supervised loss functions.
  arXiv  Detail & Related papers  (2021-04-13T04:25:54Z)
• Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo [71.59494156155309]
  Existing approaches for multi-view 3D pose estimation explicitly establish cross-view correspondences to group 2D pose detections from multiple camera views. We present our multi-view 3D pose estimation approach based on plane sweep stereo to jointly address the cross-view fusion and 3D pose reconstruction in a single shot.
  arXiv  Detail & Related papers  (2021-04-06T03:49:35Z)
• Residual Pose: A Decoupled Approach for Depth-based 3D Human Pose Estimation [18.103595280706593]
  We leverage recent advances in reliable 2D pose estimation with CNN to estimate the 3D pose of people from depth images. Our approach achieves very competitive results both in accuracy and speed on two public datasets.
  arXiv  Detail & Related papers  (2020-11-10T10:08:13Z)
• Beyond Weak Perspective for Monocular 3D Human Pose Estimation [6.883305568568084]
  We consider the task of 3D joints location and orientation prediction from a monocular video. We first infer 2D joints locations with an off-the-shelf pose estimation algorithm. We then adhere to the SMPLify algorithm which receives those initial parameters.
  arXiv  Detail & Related papers  (2020-09-14T16:23:14Z)
• HDNet: Human Depth Estimation for Multi-Person Camera-Space Localization [83.57863764231655]
  We propose the Human Depth Estimation Network (HDNet), an end-to-end framework for absolute root joint localization. A skeleton-based Graph Neural Network (GNN) is utilized to propagate features among joints. We evaluate our HDNet on the root joint localization and root-relative 3D pose estimation tasks with two benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-17T12:44:23Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.