UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture

• URL: http://arxiv.org/abs/2208.01633v1
• Date: Tue, 2 Aug 2022 17:59:54 GMT
• Title: UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture
• Authors: Hiroyasu Akada and Jian Wang and Soshi Shimada and Masaki Takahashi and Christian Theobalt and Vladislav Golyanik
• Abstract summary: UnrealEgo is a new large-scale naturalistic dataset for egocentric 3D human pose estimation. It is based on an advanced concept of eyeglasses equipped with two fisheye cameras that can be used in unconstrained environments. We propose a new benchmark method with a simple but effective idea of devising a 2D keypoint estimation module for stereo inputs to improve 3D human pose estimation.
• Score: 70.59984501516084
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present UnrealEgo, i.e., a new large-scale naturalistic dataset for egocentric 3D human pose estimation. UnrealEgo is based on an advanced concept of eyeglasses equipped with two fisheye cameras that can be used in unconstrained environments. We design their virtual prototype and attach them to 3D human models for stereo view capture. We next generate a large corpus of human motions. As a consequence, UnrealEgo is the first dataset to provide in-the-wild stereo images with the largest variety of motions among existing egocentric datasets. Furthermore, we propose a new benchmark method with a simple but effective idea of devising a 2D keypoint estimation module for stereo inputs to improve 3D human pose estimation. The extensive experiments show that our approach outperforms the previous state-of-the-art methods qualitatively and quantitatively. UnrealEgo and our source codes are available on our project web page.

Related papers

• 3D Human Pose Perception from Egocentric Stereo Videos [67.9563319914377]
  We propose a new transformer-based framework to improve egocentric stereo 3D human pose estimation. Our method is able to accurately estimate human poses even in challenging scenarios, such as crouching and sitting. We will release UnrealEgo2, UnrealEgo-RW, and trained models on our project page.
  arXiv  Detail & Related papers  (2023-12-30T21:21:54Z)
• EgoHumans: An Egocentric 3D Multi-Human Benchmark [37.375846688453514]
  We present EgoHumans, a new multi-view multi-human video benchmark to advance the state-of-the-art of egocentric human 3D pose estimation and tracking. We propose a novel 3D capture setup to construct a comprehensive egocentric multi-human benchmark in the wild. We leverage consumer-grade wearable camera-equipped glasses for the egocentric view, which enables us to capture dynamic activities like playing tennis, fencing, volleyball, etc.
  arXiv  Detail & Related papers  (2023-05-25T21:37:36Z)
• Ego-Body Pose Estimation via Ego-Head Pose Estimation [22.08240141115053]
  Estimating 3D human motion from an egocentric video sequence plays a critical role in human behavior understanding and has various applications in VR/AR. We propose a new method, Ego-Body Pose Estimation via Ego-Head Pose Estimation (EgoEgo), which decomposes the problem into two stages, connected by the head motion as an intermediate representation. This disentanglement of head and body pose eliminates the need for training datasets with paired egocentric videos and 3D human motion.
  arXiv  Detail & Related papers  (2022-12-09T02:25:20Z)
• Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning [70.75369367311897]
  3D-aware global correspondences are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies. An adversarial generator takes the garment warped by the 3D-aware flow, and the image of the target person as inputs, to synthesize the photo-realistic try-on result.
  arXiv  Detail & Related papers  (2022-11-25T12:16:21Z)
• Human Performance Capture from Monocular Video in the Wild [50.34917313325813]
  We propose a method capable of capturing the dynamic 3D human shape from a monocular video featuring challenging body poses. Our method outperforms state-of-the-art methods on an in-the-wild human video dataset 3DPW.
  arXiv  Detail & Related papers  (2021-11-29T16:32:41Z)
• SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera [97.0162841635425]
  We present a solution to egocentric 3D body pose estimation from monocular images captured from downward looking fish-eye cameras installed on the rim of a head mounted VR device. This unusual viewpoint leads to images with unique visual appearance, with severe self-occlusions and perspective distortions. We propose an encoder-decoder architecture with a novel multi-branch decoder designed to account for the varying uncertainty in 2D predictions.
  arXiv  Detail & Related papers  (2020-11-02T16:18:06Z)
• Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution [34.301501457959056]
  We propose a temporal regression network with a gated convolution module to transform 2D joints to 3D. A simple yet effective localization approach is also conducted to transform the normalized pose to the global trajectory. Our proposed method outperforms most state-of-the-art 2D-to-3D pose estimation methods.
  arXiv  Detail & Related papers  (2020-10-31T04:35:24Z)
• Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild [101.70320427145388]
  We propose a weakly-supervised approach that does not require 3D annotations and learns to estimate 3D poses from unlabeled multi-view data. We evaluate our proposed approach on two large scale datasets.
  arXiv  Detail & Related papers  (2020-03-17T08:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.