Unsupervised View-Invariant Human Posture Representation

• URL: http://arxiv.org/abs/2109.08730v2
• Date: Mon, 8 Jul 2024 13:42:17 GMT
• Title: Unsupervised View-Invariant Human Posture Representation
• Authors: Faegheh Sardari, Björn Ommer, Majid Mirmehdi,
• Abstract summary: We present a novel unsupervised approach that learns to extract view-invariant 3D human pose representation from a 2D image. Our model is trained by exploiting the intrinsic view-invariant properties of human pose between simultaneous frames. We show improvements on the state-of-the-art unsupervised cross-view action classification accuracy on RGB and depth images.
• Score: 28.840986167408037
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most recent view-invariant action recognition and performance assessment approaches rely on a large amount of annotated 3D skeleton data to extract view-invariant features. However, acquiring 3D skeleton data can be cumbersome, if not impractical, in in-the-wild scenarios. To overcome this problem, we present a novel unsupervised approach that learns to extract view-invariant 3D human pose representation from a 2D image without using 3D joint data. Our model is trained by exploiting the intrinsic view-invariant properties of human pose between simultaneous frames from different viewpoints and their equivariant properties between augmented frames from the same viewpoint. We evaluate the learned view-invariant pose representations for two downstream tasks. We perform comparative experiments that show improvements on the state-of-the-art unsupervised cross-view action classification accuracy on NTU RGB+D by a significant margin, on both RGB and depth images. We also show the efficiency of transferring the learned representations from NTU RGB+D to obtain the first ever unsupervised cross-view and cross-subject rank correlation results on the multi-view human movement quality dataset, QMAR, and marginally improve on the-state-of-the-art supervised results for this dataset. We also carry out ablation studies to examine the contributions of the different components of our proposed network.

Related papers

• Cohere3D: Exploiting Temporal Coherence for Unsupervised Representation Learning of Vision-based Autonomous Driving [73.3702076688159]
  We propose a novel contrastive learning algorithm, Cohere3D, to learn coherent instance representations in a long-term input sequence. We evaluate our algorithm by finetuning the pretrained model on various downstream perception, prediction, and planning tasks.
  arXiv  Detail & Related papers  (2024-02-23T19:43:01Z)
• RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation [50.35403070279804]
  3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. We propose RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction.
  arXiv  Detail & Related papers  (2023-12-19T03:39:56Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Occlusion-Invariant Rotation-Equivariant Semi-Supervised Depth Based Cross-View Gait Pose Estimation [40.50555832966361]
  We propose a novel approach for cross-view generalization with an occlusion-invariant semi-supervised learning framework. Our model was trained with real-world data from a single view and unlabelled synthetic data from multiple views. It can generalize well on the real-world data from all the other unseen views.
  arXiv  Detail & Related papers  (2021-09-03T09:39:05Z)
• View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose [36.384824115033304]
  We propose an approach to learning a compact view-invariant embedding space from 2D body joint keypoints, without explicitly predicting 3D poses. Experimental results show that our embedding model achieves higher accuracy when retrieving similar poses across different camera views.
  arXiv  Detail & Related papers  (2020-10-23T17:58:35Z)
• Appearance Consensus Driven Self-Supervised Human Mesh Recovery [67.20942777949793]
  We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images. We achieve state-of-the-art results on the standard model-based 3D pose estimation benchmarks. The resulting colored mesh prediction opens up the usage of our framework for a variety of appearance-related tasks beyond the pose and shape estimation.
  arXiv  Detail & Related papers  (2020-08-04T05:40:39Z)
• Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation [58.72192168935338]
  Generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable. We propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our proposed model employs three consecutive differentiable transformations named as forward-kinematics, camera-projection and spatial-map transformation.
  arXiv  Detail & Related papers  (2020-06-24T23:56:33Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.