Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey

• URL: http://arxiv.org/abs/2207.04512v1
• Date: Sun, 10 Jul 2022 18:13:25 GMT
• Title: Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey
• Authors: Seyed Mojtaba Marvasti-Zadeh, Mohammad N.S. Jahromi, Javad Khaghanix, Devin Goodsman, Nilanjan Ray, Nadir Erbilgin
• Abstract summary: In nature, the collective behavior of animals is dominated by the interactions between individuals of the same species. Recent advances in 3D vision have led to a number of impressive works on the 3D shape and pose estimation. This work is the first attempt to provide an overview of recent advances in 3D bird reconstruction based on monocular vision.
• Score: 6.555250822345809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In nature, the collective behavior of animals, such as flying birds is dominated by the interactions between individuals of the same species. However, the study of such behavior among the bird species is a complex process that humans cannot perform using conventional visual observational techniques such as focal sampling in nature. For social animals such as birds, the mechanism of group formation can help ecologists understand the relationship between social cues and their visual characteristics over time (e.g., pose and shape). But, recovering the varying pose and shapes of flying birds is a highly challenging problem. A widely-adopted solution to tackle this bottleneck is to extract the pose and shape information from 2D image to 3D correspondence. Recent advances in 3D vision have led to a number of impressive works on the 3D shape and pose estimation, each with different pros and cons. To the best of our knowledge, this work is the first attempt to provide an overview of recent advances in 3D bird reconstruction based on monocular vision, give both computer vision and biology researchers an overview of existing approaches, and compare their characteristics.

Related papers

• Temporally-consistent 3D Reconstruction of Birds [5.787285686300833]
  We propose an approach to reconstruct the 3D pose and shape from monocular videos of a specific breed of seabird - the common murre. We provide a real-world dataset of 10000 frames of video observations on average capture nine birds simultaneously.
  arXiv  Detail & Related papers  (2024-08-24T17:12:36Z)
• Learning the 3D Fauna of the Web [70.01196719128912]
  We develop 3D-Fauna, an approach that learns a pan-category deformable 3D animal model for more than 100 animal species jointly. One crucial bottleneck of modeling animals is the limited availability of training data. We show that prior category-specific attempts fail to generalize to rare species with limited training images.
  arXiv  Detail & Related papers  (2024-01-04T18:32:48Z)
• Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image [30.997936022365018]
  We propose the combination of two-stage supervised and self-supervised training to address the challenge of obtaining animal cooperation for 3D scanning. Results of our study demonstrate that our approach outperforms state-of-the-art methods in both quantitative and qualitative aspects of bird 3D digitization.
  arXiv  Detail & Related papers  (2023-11-22T07:06:38Z)
• Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary [32.19504891200443]
  We introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers.
  arXiv  Detail & Related papers  (2022-12-01T04:23:18Z)
• MagicPony: Learning Articulated 3D Animals in the Wild [81.63322697335228]
  We present a new method, dubbed MagicPony, that learns this predictor purely from in-the-wild single-view images of the object category. At its core is an implicit-explicit representation of articulated shape and appearance, combining the strengths of neural fields and meshes.
  arXiv  Detail & Related papers  (2022-11-22T18:59:31Z)
• State of the Art in Dense Monocular Non-Rigid 3D Reconstruction [100.9586977875698]
  3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views.
  arXiv  Detail & Related papers  (2022-10-27T17:59:53Z)
• LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery [72.3681707384754]
  We propose a practical problem setting to estimate 3D pose and shape of animals given only a few in-the-wild images of a particular animal species. We do not assume any form of 2D or 3D ground-truth annotations, nor do we leverage any multi-view or temporal information. Following these insights, we propose LASSIE, a novel optimization framework which discovers 3D parts in a self-supervised manner.
  arXiv  Detail & Related papers  (2022-07-07T17:00:07Z)
• DOVE: Learning Deformable 3D Objects by Watching Videos [89.43105063468077]
  We present DOVE, which learns to predict 3D canonical shape, deformation, viewpoint and texture from a single 2D image of a bird. Our method reconstructs temporally consistent 3D shape and deformation, which allows us to animate and re-render the bird from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2021-07-22T17:58:10Z)
• Birds of a Feather: Capturing Avian Shape Models from Images [46.84613650829397]
  We present a method to capture new species using an articulated template and images of that species. We learn a shape space that captures variation both among species and within each species from image evidence. Using a low-dimensional embedding, we show that our learned 3D shape space better reflects the phylogenetic relationships among birds than learned perceptual features.
  arXiv  Detail & Related papers  (2021-05-19T20:53:48Z)
• 3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View [35.61330221535231]
  We introduce a model and multi-view optimization approach to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. We provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary.
  arXiv  Detail & Related papers  (2020-08-13T23:29:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.