Learning from Abstract Images: on the Importance of Occlusion in a Minimalist Encoding of Human Poses

• URL: http://arxiv.org/abs/2307.09893v1
• Date: Wed, 19 Jul 2023 10:45:49 GMT
• Title: Learning from Abstract Images: on the Importance of Occlusion in a Minimalist Encoding of Human Poses
• Authors: Saad Manzur, Wayne Hayes
• Abstract summary: 2D-to-D representation suffers from poor performance in cross-dataset benchmarks. We propose a novel representation using 3D information while encoding it. The result allows us to predict poses that are completely independent of camera viewpoint.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing 2D-to-3D pose lifting networks suffer from poor performance in cross-dataset benchmarks. Although the use of 2D keypoints joined by "stick-figure" limbs has shown promise as an intermediate step, stick-figures do not account for occlusion information that is often inherent in an image. In this paper, we propose a novel representation using opaque 3D limbs that preserves occlusion information while implicitly encoding joint locations. Crucially, when training on data with accurate three-dimensional keypoints and without part-maps, this representation allows training on abstract synthetic images, with occlusion, from as many synthetic viewpoints as desired. The result is a pose defined by limb angles rather than joint positions $\unicode{x2013}$ because poses are, in the real world, independent of cameras $\unicode{x2013}$ allowing us to predict poses that are completely independent of camera viewpoint. The result provides not only an improvement in same-dataset benchmarks, but a "quantum leap" in cross-dataset benchmarks.

Related papers

• PostoMETRO: Pose Token Enhanced Mesh Transformer for Robust 3D Human Mesh Recovery [20.763457281944834]
  We present PostoMETRO, which integrates 2D pose representation into transformers in a token-wise manner. We are able to produce more precise 3D coordinates, even under extreme scenarios like occlusion.
  arXiv  Detail & Related papers  (2024-03-19T06:18:25Z)
• Occlusion Resilient 3D Human Pose Estimation [52.49366182230432]
  Occlusions remain one of the key challenges in 3D body pose estimation from single-camera video sequences. We demonstrate the effectiveness of this approach compared to state-of-the-art techniques that infer poses from single-camera sequences.
  arXiv  Detail & Related papers  (2024-02-16T19:29:43Z)
• A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation [18.72362803593654]
  The dominant paradigm in 3D human pose estimation that lifts a 2D pose sequence to 3D heavily relies on long-term temporal clues. This can be attributed to their inherent inability to perceive spatial context as plain 2D joint coordinates carry no visual cues. We propose a straightforward yet powerful solution: leveraging the readily available intermediate visual representations produced by off-the-shelf (pre-trained) 2D pose detectors.
  arXiv  Detail & Related papers  (2023-11-06T18:04:13Z)
• Improving 3D Pose Estimation for Sign Language [38.20064386142944]
  This work addresses 3D human pose reconstruction in single images. We present a method that combines Forward Kinematics (FK) with neural networks to ensure a fast and valid prediction of 3D pose.
  arXiv  Detail & Related papers  (2023-08-18T13:05:10Z)
• Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens [34.50928515515274]
  We present a novel method to estimate 3D human pose and shape from monocular videos. The proposed method attains superior performances on the 3DPW and Human3.6M datasets.
  arXiv  Detail & Related papers  (2023-03-01T07:48:01Z)
• Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images [133.68032636906133]
  We study the problem of estimating 3D shape and pose of an object in terms of keypoints, from a single 2D image. The shape and pose are learned directly from images collected by categories and their partial 2D keypoint annotations.
  arXiv  Detail & Related papers  (2022-11-14T16:18:11Z)
• Learning Visibility for Robust Dense Human Body Estimation [78.37389398573882]
  Estimating 3D human pose and shape from 2D images is a crucial yet challenging task. We learn dense human body estimation that is robust to partial observations. We obtain pseudo ground-truths of visibility labels from dense UV correspondences and train a neural network to predict visibility along with 3D coordinates.
  arXiv  Detail & Related papers  (2022-08-23T00:01:05Z)
• VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild [98.69191256693703]
  We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.
  arXiv  Detail & Related papers  (2021-08-05T08:35:44Z)
• AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild [77.43884383743872]
  We present AdaFuse, an adaptive multiview fusion method to enhance the features in occluded views. We extensively evaluate the approach on three public datasets including Human3.6M, Total Capture and CMU Panoptic. We also create a large scale synthetic dataset Occlusion-Person, which allows us to perform numerical evaluation on the occluded joints.
  arXiv  Detail & Related papers  (2020-10-26T03:19:46Z)
• Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach [76.10879433430466]
  We propose to estimate 3D human pose from multi-view images and a few IMUs attached at person's limbs. It operates by firstly detecting 2D poses from the two signals, and then lifting them to the 3D space. The simple two-step approach reduces the error of the state-of-the-art by a large margin on a public dataset.
  arXiv  Detail & Related papers  (2020-03-25T00:26:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.