Related papers: 3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View

3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View

URL: http://arxiv.org/abs/2008.06133v1
Date: Thu, 13 Aug 2020 23:29:04 GMT
Title: 3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View
Authors: Marc Badger, Yufu Wang, Adarsh Modh, Ammon Perkes, Nikos Kolotouros, Bernd G. Pfrommer, Marc F. Schmidt, Kostas Daniilidis
Abstract summary: We introduce a model and multi-view optimization approach to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. We provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary.
Score: 35.61330221535231
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Automated capture of animal pose is transforming how we study neuroscience and social behavior. Movements carry important social cues, but current methods are not able to robustly estimate pose and shape of animals, particularly for social animals such as birds, which are often occluded by each other and objects in the environment. To address this problem, we first introduce a model and multi-view optimization approach, which we use to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. Finally, we provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary. The project website with videos, results, code, mesh model, and the Penn Aviary Dataset can be found at https://marcbadger.github.io/avian-mesh.

Related papers

Temporally-consistent 3D Reconstruction of Birds [5.787285686300833]
We propose an approach to reconstruct the 3D pose and shape from monocular videos of a specific breed of seabird - the common murre. We provide a real-world dataset of 10000 frames of video observations on average capture nine birds simultaneously.
arXiv Detail & Related papers (2024-08-24T17:12:36Z)
Learning the 3D Fauna of the Web [70.01196719128912]
We develop 3D-Fauna, an approach that learns a pan-category deformable 3D animal model for more than 100 animal species jointly. One crucial bottleneck of modeling animals is the limited availability of training data. We show that prior category-specific attempts fail to generalize to rare species with limited training images.
arXiv Detail & Related papers (2024-01-04T18:32:48Z)
Multi-view Tracking, Re-ID, and Social Network Analysis of a Flock of Visually Similar Birds in an Outdoor Aviary [32.19504891200443]
We introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers.
arXiv Detail & Related papers (2022-12-01T04:23:18Z)
MagicPony: Learning Articulated 3D Animals in the Wild [81.63322697335228]
We present a new method, dubbed MagicPony, that learns this predictor purely from in-the-wild single-view images of the object category. At its core is an implicit-explicit representation of articulated shape and appearance, combining the strengths of neural fields and meshes.
arXiv Detail & Related papers (2022-11-22T18:59:31Z)
Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Survey [6.555250822345809]
In nature, the collective behavior of animals is dominated by the interactions between individuals of the same species. Recent advances in 3D vision have led to a number of impressive works on the 3D shape and pose estimation. This work is the first attempt to provide an overview of recent advances in 3D bird reconstruction based on monocular vision.
arXiv Detail & Related papers (2022-07-10T18:13:25Z)
Neural Rendering of Humans in Novel View and Pose from Monocular Video [68.37767099240236]
We introduce a new method that generates photo-realistic humans under novel views and poses given a monocular video as input. Our method significantly outperforms existing approaches under unseen poses and novel views given monocular videos as input.
arXiv Detail & Related papers (2022-04-04T03:09:20Z)
Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise [29.52068540448424]
We propose an end-to-end unified 3D mesh recovery of humans and quadruped animals trained in a weakly-supervised way. We exploit the morphological similarity between humans and animals, motivated by animal exercise where humans imitate animal poses.
arXiv Detail & Related papers (2021-11-03T18:15:50Z)
Unsupervised Shape and Pose Disentanglement for 3D Meshes [49.431680543840706]
We present a simple yet effective approach to learn disentangled shape and pose representations in an unsupervised setting. We use a combination of self-consistency and cross-consistency constraints to learn pose and shape space from registered meshes. We demonstrate the usefulness of learned representations through a number of tasks including pose transfer and shape retrieval.
arXiv Detail & Related papers (2020-07-22T11:00:27Z)
Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals [56.65062746564091]
We aim to capture the pose of neuroscience model organisms, without using any manual supervision, to study how neural circuits orchestrate behaviour. Our key contribution is the explicit and independent modeling of appearance, shape and poses in an unpaired image translation framework. We demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish)
arXiv Detail & Related papers (2020-01-23T15:34:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.