BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information

• URL: http://arxiv.org/abs/2203.15536v2
• Date: Wed, 30 Mar 2022 19:32:59 GMT
• Title: BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information
• Authors: Nadine Rueegg, Silvia Zuffi, Konrad Schindler and Michael J. Black
• Abstract summary: Our goal is to recover the 3D shape and pose of dogs from a single image. Recent work has proposed to directly regress the SMAL animal model, with additional limb scale parameters, from images. Our method, called BARC (Breed-Augmented Regression using Classification), goes beyond prior work in several important ways. This work shows that a-priori information about genetic similarity can help to compensate for the lack of 3D training data.
• Score: 66.77206206569802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our goal is to recover the 3D shape and pose of dogs from a single image. This is a challenging task because dogs exhibit a wide range of shapes and appearances, and are highly articulated. Recent work has proposed to directly regress the SMAL animal model, with additional limb scale parameters, from images. Our method, called BARC (Breed-Augmented Regression using Classification), goes beyond prior work in several important ways. First, we modify the SMAL shape space to be more appropriate for representing dog shape. But, even with a better shape model, the problem of regressing dog shape from an image is still challenging because we lack paired images with 3D ground truth. To compensate for the lack of paired data, we formulate novel losses that exploit information about dog breeds. In particular, we exploit the fact that dogs of the same breed have similar body shapes. We formulate a novel breed similarity loss consisting of two parts: One term encourages the shape of dogs from the same breed to be more similar than dogs of different breeds. The second one, a breed classification loss, helps to produce recognizable breed-specific shapes. Through ablation studies, we find that our breed losses significantly improve shape accuracy over a baseline without them. We also compare BARC qualitatively to WLDO with a perceptual study and find that our approach produces dogs that are significantly more realistic. This work shows that a-priori information about genetic similarity can help to compensate for the lack of 3D training data. This concept may be applicable to other animal species or groups of species. Our code is publicly available for research purposes at https://barc.is.tue.mpg.de/.

Related papers

• ShapeBoost: Boosting Human Shape Estimation with Part-Based Parameterization and Clothing-Preserving Augmentation [58.50613393500561]
  We propose ShapeBoost, a new human shape recovery framework. It achieves pixel-level alignment even for rare body shapes and high accuracy for people wearing different types of clothes.
  arXiv  Detail & Related papers  (2024-03-02T23:40:23Z)
• Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR [103.51937218213774]
  This paper advances the fine-grained sketch-based image retrieval (FG-SBIR) literature by putting forward a strong baseline that overshoots prior state-of-the-arts by 11%. We propose a simple modification to the standard triplet loss, that explicitly enforces separation amongst photos/sketch instances. For (i) we employ an intra-modal triplet loss amongst sketches to bring sketches of the same instance closer from others, and one more amongst photos to push away different photo instances.
  arXiv  Detail & Related papers  (2023-03-24T03:34:33Z)
• A Kendall Shape Space Approach to 3D Shape Estimation from 2D Landmarks [0.5161531917413708]
  We present a new approach based on Kendall's shape space to reconstruct 3D shapes from single monocular 2D images. The work is motivated by an application to study the feeding behavior of the basking shark.
  arXiv  Detail & Related papers  (2022-07-26T07:00:50Z)
• Accurate 3D Body Shape Regression using Metric and Semantic Attributes [55.58629009876271]
  We show that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes. This is the first demonstration that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes.
  arXiv  Detail & Related papers  (2022-06-14T17:54:49Z)
• Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise [29.52068540448424]
  We propose an end-to-end unified 3D mesh recovery of humans and quadruped animals trained in a weakly-supervised way. We exploit the morphological similarity between humans and animals, motivated by animal exercise where humans imitate animal poses.
  arXiv  Detail & Related papers  (2021-11-03T18:15:50Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• Unsupervised Shape and Pose Disentanglement for 3D Meshes [49.431680543840706]
  We present a simple yet effective approach to learn disentangled shape and pose representations in an unsupervised setting. We use a combination of self-consistency and cross-consistency constraints to learn pose and shape space from registered meshes. We demonstrate the usefulness of learned representations through a number of tasks including pose transfer and shape retrieval.
  arXiv  Detail & Related papers  (2020-07-22T11:00:27Z)
• Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop [25.40930904714051]
  We introduce an automatic, end-to-end method for recovering the 3D pose and shape of dogs from monocular internet images. We learn a richer prior over shapes than previous work, which helps regularize parameter estimation. We demonstrate results on the Stanford Dog dataset, an 'in the wild' dataset of 20,580 dog images.
  arXiv  Detail & Related papers  (2020-07-21T21:52:56Z)
• RGBD-Dog: Predicting Canine Pose from RGBD Sensors [25.747221533627464]
  We focus on the problem of 3D canine pose estimation from RGBD images. We generate a dataset of synthetic RGBD images from this data. A stacked hourglass network is trained to predict 3D joint locations.
  arXiv  Detail & Related papers  (2020-04-16T17:34:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.