RGBD-Dog: Predicting Canine Pose from RGBD Sensors

• URL: http://arxiv.org/abs/2004.07788v1
• Date: Thu, 16 Apr 2020 17:34:45 GMT
• Title: RGBD-Dog: Predicting Canine Pose from RGBD Sensors
• Authors: Sinead Kearney, Wenbin Li, Martin Parsons, Kwang In Kim, Darren Cosker
• Abstract summary: We focus on the problem of 3D canine pose estimation from RGBD images. We generate a dataset of synthetic RGBD images from this data. A stacked hourglass network is trained to predict 3D joint locations.
• Score: 25.747221533627464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The automatic extraction of animal \reb{3D} pose from images without markers is of interest in a range of scientific fields. Most work to date predicts animal pose from RGB images, based on 2D labelling of joint positions. However, due to the difficult nature of obtaining training data, no ground truth dataset of 3D animal motion is available to quantitatively evaluate these approaches. In addition, a lack of 3D animal pose data also makes it difficult to train 3D pose-prediction methods in a similar manner to the popular field of body-pose prediction. In our work, we focus on the problem of 3D canine pose estimation from RGBD images, recording a diverse range of dog breeds with several Microsoft Kinect v2s, simultaneously obtaining the 3D ground truth skeleton via a motion capture system. We generate a dataset of synthetic RGBD images from this data. A stacked hourglass network is trained to predict 3D joint locations, which is then constrained using prior models of shape and pose. We evaluate our model on both synthetic and real RGBD images and compare our results to previously published work fitting canine models to images. Finally, despite our training set consisting only of dog data, visual inspection implies that our network can produce good predictions for images of other quadrupeds -- e.g. horses or cats -- when their pose is similar to that contained in our training set.

Related papers

• ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data [0.2302001830524133]
  We use synthetic data generated with a 3D simulator to obtain the first synthetic dataset that can be used for both detection and 2D pose estimation of zebras. We extensively train and benchmark our detection and 2D pose estimation models on multiple real-world and synthetic datasets. These experiments show how the models trained from scratch and only with synthetic data can consistently generalize to real-world images of zebras.
  arXiv  Detail & Related papers  (2024-08-20T13:28:37Z)
• MPL: Lifting 3D Human Pose from Multi-view 2D Poses [75.26416079541723]
  We propose combining 2D pose estimation, for which large and rich training datasets exist, and 2D-to-3D pose lifting, using a transformer-based network. Our experiments demonstrate decreases up to 45% in MPJPE errors compared to the 3D pose obtained by triangulating the 2D poses.
  arXiv  Detail & Related papers  (2024-08-20T12:55:14Z)
• Benchmarking Monocular 3D Dog Pose Estimation Using In-The-Wild Motion Capture Data [17.042955091063444]
  We introduce a new benchmark analysis focusing on 3D canine pose estimation from monocular in-the-wild images. A multi-modal dataset 3DDogs-Lab was captured indoors, featuring various dog breeds trotting on a walkway. We create 3DDogs-Wild, a naturalised version of the dataset where the optical markers are in-painted and the subjects are placed in diverse environments. We show that using the 3DDogs-Wild to train the models leads to improved performance when evaluating on in-the-wild data.
  arXiv  Detail & Related papers  (2024-06-20T15:33:39Z)
• LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery [72.3681707384754]
  We propose a practical problem setting to estimate 3D pose and shape of animals given only a few in-the-wild images of a particular animal species. We do not assume any form of 2D or 3D ground-truth annotations, nor do we leverage any multi-view or temporal information. Following these insights, we propose LASSIE, a novel optimization framework which discovers 3D parts in a self-supervised manner.
  arXiv  Detail & Related papers  (2022-07-07T17:00:07Z)
• AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation [24.009674750548303]
  Testing a pre-trained 3D pose estimator on a new dataset results in a major performance drop. We propose AdaptPose, an end-to-end framework that generates synthetic 3D human motions from a source dataset. Our method outperforms previous work in cross-dataset evaluations by 14% and previous semi-supervised learning methods that use partial 3D annotations by 16%.
  arXiv  Detail & Related papers  (2021-12-22T00:27:52Z)
• CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields [67.76151996543588]
  We learn a 3D- and camera-aware generative model which faithfully recovers not only the image but also the camera data distribution. At test time, our model generates images with explicit control over the camera as well as the shape and appearance of the scene.
  arXiv  Detail & Related papers  (2021-03-31T17:59:24Z)
• Real-time RGBD-based Extended Body Pose Estimation [57.61868412206493]
  We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation. We train estimators of body pose and facial expression parameters.
  arXiv  Detail & Related papers  (2021-03-05T13:37:50Z)
• Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop [25.40930904714051]
  We introduce an automatic, end-to-end method for recovering the 3D pose and shape of dogs from monocular internet images. We learn a richer prior over shapes than previous work, which helps regularize parameter estimation. We demonstrate results on the Stanford Dog dataset, an 'in the wild' dataset of 20,580 dog images.
  arXiv  Detail & Related papers  (2020-07-21T21:52:56Z)
• Towards Generalization of 3D Human Pose Estimation In The Wild [73.19542580408971]
  3DBodyTex.Pose is a dataset that addresses the task of 3D human pose estimation in-the-wild. 3DBodyTex.Pose offers high quality and rich data containing 405 different real subjects in various clothing and poses, and 81k image samples with ground-truth 2D and 3D pose annotations.
  arXiv  Detail & Related papers  (2020-04-21T13:31:58Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.