Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

• URL: http://arxiv.org/abs/2108.07777v1
• Date: Tue, 17 Aug 2021 17:31:24 GMT
• Title: Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry
• Authors: Arij Bouazizi, Julian Wiederer, Ulrich Kressel and Vasileios Belagiannis
• Abstract summary: We present a self-supervised learning algorithm for 3D human pose estimation of a single person based on a multiple-view camera system. We propose a four-loss function learning algorithm, which does not require any 2D or 3D body pose ground-truth.
• Score: 2.7541825072548805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a self-supervised learning algorithm for 3D human pose estimation of a single person based on a multiple-view camera system and 2D body pose estimates for each view. To train our model, represented by a deep neural network, we propose a four-loss function learning algorithm, which does not require any 2D or 3D body pose ground-truth. The proposed loss functions make use of the multiple-view geometry to reconstruct 3D body pose estimates and impose body pose constraints across the camera views. Our approach utilizes all available camera views during training, while the inference is single-view. In our evaluations, we show promising performance on Human3.6M and HumanEva benchmarks, while we also present a generalization study on MPI-INF-3DHP dataset, as well as several ablation results. Overall, we outperform all self-supervised learning methods and reach comparable results to supervised and weakly-supervised learning approaches. Our code and models are publicly available

Related papers

• Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos [15.532504015622159]
  Category-level 3D pose estimation is a fundamentally important problem in computer vision and robotics. We tackle the problem of learning to estimate the category-level 3D pose only from casually taken object-centric videos.
  arXiv  Detail & Related papers  (2024-07-05T09:43:05Z)
• Self-learning Canonical Space for Multi-view 3D Human Pose Estimation [57.969696744428475]
  Multi-view 3D human pose estimation is naturally superior to single view one. The accurate annotation of these information is hard to obtain. We propose a fully self-supervised framework, named cascaded multi-view aggregating network (CMANet) CMANet is superior to state-of-the-art methods in extensive quantitative and qualitative analysis.
  arXiv  Detail & Related papers  (2024-03-19T04:54:59Z)
• Self-supervised 3D Human Pose Estimation from a Single Image [1.0878040851638]
  We propose a new self-supervised method for predicting 3D human body pose from a single image. The prediction network is trained from a dataset of unlabelled images depicting people in typical poses and a set of unpaired 2D poses.
  arXiv  Detail & Related papers  (2023-04-05T10:26:21Z)
• Learning Temporal 3D Human Pose Estimation with Pseudo-Labels [3.0954251281114513]
  We present a simple, yet effective, approach for self-supervised 3D human pose estimation. We rely on triangulating 2D body pose estimates of a multiple-view camera system. Our method achieves state-of-the-art performance in the Human3.6M and MPI-INF-3DHP benchmarks.
  arXiv  Detail & Related papers  (2021-10-14T17:40:45Z)
• VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild [98.69191256693703]
  We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.
  arXiv  Detail & Related papers  (2021-08-05T08:35:44Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks [53.71440550507745]
  Reconstructing 3D models from 2D images is one of the fundamental problems in computer vision. We propose a deep learning technique for 3D object reconstruction from a single image. We learn both 3D point cloud reconstruction and pose estimation networks in a self-supervised manner.
  arXiv  Detail & Related papers  (2020-05-05T04:25:16Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Multi-Person Absolute 3D Human Pose Estimation with Weak Depth Supervision [0.0]
  We introduce a network that can be trained with additional RGB-D images in a weakly supervised fashion. Our algorithm is a monocular, multi-person, absolute pose estimator. We evaluate the algorithm on several benchmarks, showing a consistent improvement in error rates.
  arXiv  Detail & Related papers  (2020-04-08T13:29:22Z)
• Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild [101.70320427145388]
  We propose a weakly-supervised approach that does not require 3D annotations and learns to estimate 3D poses from unlabeled multi-view data. We evaluate our proposed approach on two large scale datasets.
  arXiv  Detail & Related papers  (2020-03-17T08:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.