Occlusion Robust 3D Human Pose Estimation with StridedPoseGraphFormer and Data Augmentation

• URL: http://arxiv.org/abs/2304.12069v1
• Date: Mon, 24 Apr 2023 13:05:13 GMT
• Title: Occlusion Robust 3D Human Pose Estimation with StridedPoseGraphFormer and Data Augmentation
• Authors: Soubarna Banik, Patricia Gscho{\ss}mann, Alejandro Mendoza Garcia, Alois Knoll
• Abstract summary: We show that our proposed method compares favorably with the state-of-the-art (SoA) Our experimental results also reveal that in the absence of any occlusion handling mechanism, the performance of SoA 3D HPE methods degrades significantly when they encounter occlusion.
• Score: 69.49430149980789
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Occlusion is an omnipresent challenge in 3D human pose estimation (HPE). In spite of the large amount of research dedicated to 3D HPE, only a limited number of studies address the problem of occlusion explicitly. To fill this gap, we propose to combine exploitation of spatio-temporal features with synthetic occlusion augmentation during training to deal with occlusion. To this end, we build a spatio-temporal 3D HPE model, StridedPoseGraphFormer based on graph convolution and transformers, and train it using occlusion augmentation. Unlike the existing occlusion-aware methods, that are only tested for limited occlusion, we extensively evaluate our method for varying degrees of occlusion. We show that our proposed method compares favorably with the state-of-the-art (SoA). Our experimental results also reveal that in the absence of any occlusion handling mechanism, the performance of SoA 3D HPE methods degrades significantly when they encounter occlusion.

Related papers

• Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual Transformer Fusion [13.938406073551844]
  This paper introduces a Dual Transformer Fusion (DTF) algorithm, a novel approach to obtain a holistic 3D pose estimation. To enable precise 3D Human Pose Estimation, our approach leverages the innovative DTF architecture, which first generates a pair of intermediate views. Our approach outperforms existing state-of-the-art methods on both datasets, yielding substantial improvements.
  arXiv  Detail & Related papers  (2024-10-06T18:15:27Z)
• 3D Human Pose Estimation with Occlusions: Introducing BlendMimic3D Dataset and GCN Refinement [6.858859328420893]
  This work identifies and addresses a gap in the current state of the art in 3D Human Pose Estimation (HPE) We introduce our novel BlendMimic3D dataset, designed to mimic real-world situations where occlusions occur. We also propose a 3D pose refinement block, employing a Graph Convolutional Network (GCN) to enhance pose representation through a graph model.
  arXiv  Detail & Related papers  (2024-04-24T18:49:37Z)
• OccGaussian: 3D Gaussian Splatting for Occluded Human Rendering [55.50438181721271]
  Previous method utilizing NeRF for surface rendering to recover the occluded areas requires more than one day to train and several seconds to render occluded areas. We propose OccGaussian based on 3D Gaussian Splatting, which can be trained within 6 minutes and produces high-quality human renderings up to 160 FPS with occluded input.
  arXiv  Detail & Related papers  (2024-04-12T13:00:06Z)
• Occlusion Resilient 3D Human Pose Estimation [52.49366182230432]
  Occlusions remain one of the key challenges in 3D body pose estimation from single-camera video sequences. We demonstrate the effectiveness of this approach compared to state-of-the-art techniques that infer poses from single-camera sequences.
  arXiv  Detail & Related papers  (2024-02-16T19:29:43Z)
• 3D Textured Shape Recovery with Learned Geometric Priors [58.27543892680264]
  This technical report presents our approach to address limitations by incorporating learned geometric priors. We generate a SMPL model from learned pose prediction and fuse it into the partial input to add prior knowledge of human bodies. We also propose a novel completeness-aware bounding box adaptation for handling different levels of scales.
  arXiv  Detail & Related papers  (2022-09-07T16:03:35Z)
• On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation [57.766049538913926]
  Supervised approaches to 3D pose estimation from single images are remarkably effective when labeled data is abundant. Much of the recent attention has shifted towards semi and (or) weakly supervised learning. We propose to impose multi-view geometrical constraints by means of a differentiable triangulation and to use it as form of self-supervision during training when no labels are available.
  arXiv  Detail & Related papers  (2022-03-29T19:11:54Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework [11.509692423756448]
  The proposed method estimates more accurate 3D human poses using 2D skeletons with missing joints as input. Our experiments demonstrate the effectiveness of the proposed framework for handling the missing joints.
  arXiv  Detail & Related papers  (2022-03-08T14:35:46Z)
• 3D Pose Detection in Videos: Focusing on Occlusion [0.4588028371034406]
  We build upon existing methods for occlusion-aware 3D pose detection in videos. We implement a two stage architecture that consists of the stacked hourglass network to produce 2D pose predictions. To facilitate prediction on poses with occluded joints, we introduce an intuitive generalization of the cylinder man model.
  arXiv  Detail & Related papers  (2020-06-24T07:01:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.