Birds of a Feather: Capturing Avian Shape Models from Images

• URL: http://arxiv.org/abs/2105.09396v1
• Date: Wed, 19 May 2021 20:53:48 GMT
• Title: Birds of a Feather: Capturing Avian Shape Models from Images
• Authors: Yufu Wang, Nikos Kolotouros, Kostas Daniilidis, Marc Badger
• Abstract summary: We present a method to capture new species using an articulated template and images of that species. We learn a shape space that captures variation both among species and within each species from image evidence. Using a low-dimensional embedding, we show that our learned 3D shape space better reflects the phylogenetic relationships among birds than learned perceptual features.
• Score: 46.84613650829397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Animals are diverse in shape, but building a deformable shape model for a new species is not always possible due to the lack of 3D data. We present a method to capture new species using an articulated template and images of that species. In this work, we focus mainly on birds. Although birds represent almost twice the number of species as mammals, no accurate shape model is available. To capture a novel species, we first fit the articulated template to each training sample. By disentangling pose and shape, we learn a shape space that captures variation both among species and within each species from image evidence. We learn models of multiple species from the CUB dataset, and contribute new species-specific and multi-species shape models that are useful for downstream reconstruction tasks. Using a low-dimensional embedding, we show that our learned 3D shape space better reflects the phylogenetic relationships among birds than learned perceptual features.

Related papers

• AWOL: Analysis WithOut synthesis using Language [57.31874938870305]
  We leverage language to control existing 3D shape models to produce novel shapes. We show that we can use text to generate new animals not present during training. This work also constitutes the first language-driven method for generating 3D trees.
  arXiv  Detail & Related papers  (2024-04-03T20:04:44Z)
• Learning the 3D Fauna of the Web [70.01196719128912]
  We develop 3D-Fauna, an approach that learns a pan-category deformable 3D animal model for more than 100 animal species jointly. One crucial bottleneck of modeling animals is the limited availability of training data. We show that prior category-specific attempts fail to generalize to rare species with limited training images.
  arXiv  Detail & Related papers  (2024-01-04T18:32:48Z)
• Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey [6.555250822345809]
  In nature, the collective behavior of animals is dominated by the interactions between individuals of the same species. Recent advances in 3D vision have led to a number of impressive works on the 3D shape and pose estimation. This work is the first attempt to provide an overview of recent advances in 3D bird reconstruction based on monocular vision.
  arXiv  Detail & Related papers  (2022-07-10T18:13:25Z)
• Recognition of Unseen Bird Species by Learning from Field Guides [23.137536032163855]
  We exploit field guides to learn bird species recognition, in particular zero-shot recognition of unseen species. We study two approaches: (1) a contrastive encoding of illustrations, which can be fed into standard zero-shot learning schemes; and (2) a novel method that leverages the fact that illustrations are also images. Our results show that illustrations from field guides, which are readily available for a wide range of species, are indeed a competitive source of side information for zero-shot learning.
  arXiv  Detail & Related papers  (2022-06-03T09:13:46Z)
• Multi-Category Mesh Reconstruction From Image Collections [90.24365811344987]
  We present an alternative approach that infers the textured mesh of objects combining a series of deformable 3D models and a set of instance-specific deformation, pose, and texture. Our method is trained with images of multiple object categories using only foreground masks and rough camera poses as supervision. Experiments show that the proposed framework can distinguish between different object categories and learn category-specific shape priors in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-21T16:32:31Z)
• DOVE: Learning Deformable 3D Objects by Watching Videos [89.43105063468077]
  We present DOVE, which learns to predict 3D canonical shape, deformation, viewpoint and texture from a single 2D image of a bird. Our method reconstructs temporally consistent 3D shape and deformation, which allows us to animate and re-render the bird from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2021-07-22T17:58:10Z)
• 3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View [35.61330221535231]
  We introduce a model and multi-view optimization approach to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. We provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary.
  arXiv  Detail & Related papers  (2020-08-13T23:29:04Z)
• Unsupervised Shape and Pose Disentanglement for 3D Meshes [49.431680543840706]
  We present a simple yet effective approach to learn disentangled shape and pose representations in an unsupervised setting. We use a combination of self-consistency and cross-consistency constraints to learn pose and shape space from registered meshes. We demonstrate the usefulness of learned representations through a number of tasks including pose transfer and shape retrieval.
  arXiv  Detail & Related papers  (2020-07-22T11:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.