Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds

• URL: http://arxiv.org/abs/2505.16633v1
• Date: Thu, 22 May 2025 13:04:24 GMT
• Title: Towards Texture- And Shape-Independent 3D Keypoint Estimation in Birds
• Authors: Valentin Schmuker, Alex Hoi Hang Chan, Bastian Goldluecke, Urs Waldmann,
• Abstract summary: We present a texture-independent approach to estimate and track 3D joint positions of multiple pigeons.<n>We build upon the existing 3D-MuPPET framework, which estimates and tracks the 3D poses of up to 10 pigeons.
• Score: 1.837431956557716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present a texture-independent approach to estimate and track 3D joint positions of multiple pigeons. For this purpose, we build upon the existing 3D-MuPPET framework, which estimates and tracks the 3D poses of up to 10 pigeons using a multi-view camera setup. We extend this framework by using a segmentation method that generates silhouettes of the individuals, which are then used to estimate 2D keypoints. Following 3D-MuPPET, these 2D keypoints are triangulated to infer 3D poses, and identities are matched in the first frame and tracked in 2D across subsequent frames. Our proposed texture-independent approach achieves comparable accuracy to the original texture-dependent 3D-MuPPET framework. Additionally, we explore our approach's applicability to other bird species. To do that, we infer the 2D joint positions of four bird species without additional fine-tuning the model trained on pigeons and obtain preliminary promising results. Thus, we think that our approach serves as a solid foundation and inspires the development of more robust and accurate texture-independent pose estimation frameworks.

Related papers

• L3D-Pose: Lifting Pose for 3D Avatars from a Single Camera in the Wild [15.174438063000453]
  3D pose estimation provides a more comprehensive solution by incorporating depth, yet creating 3D pose datasets for animals is challenging due to their dynamic and unpredictable behaviours in natural settings.<n>We propose a framework with systematically synthesized datasets for lifting poses from 2D to 3D and then utilize this to re-target motion from wild settings onto arbitrary avatars.
  arXiv  Detail & Related papers  (2025-01-02T10:04:12Z)
• UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
  UPose3D is a novel approach for multi-view 3D human pose estimation. It improves robustness and flexibility without requiring direct 3D annotations.
  arXiv  Detail & Related papers  (2024-04-23T00:18:00Z)
• SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers [57.46911575980854]
  We introduce SkelFormer, a novel markerless motion capture pipeline for multi-view human pose and shape estimation. Our method first uses off-the-shelf 2D keypoint estimators, pre-trained on large-scale in-the-wild data, to obtain 3D joint positions. Next, we design a regression-based inverse-kinematic skeletal transformer that maps the joint positions to pose and shape representations from heavily noisy observations.
  arXiv  Detail & Related papers  (2024-04-19T04:51:18Z)
• 3D-MuPPET: 3D Multi-Pigeon Pose Estimation and Tracking [14.52333427647304]
  We present 3D-MuPPET, a framework to estimate and track 3D poses of up to 10 pigeons at interactive speed using multiple camera views. For identity matching, we first dynamically match 2D detections to global identities in the first frame, then use a 2D tracker to maintain IDs across views in subsequent frames. We show that 3D-MuPPET also works in outdoors without additional annotations from natural environments.
  arXiv  Detail & Related papers  (2023-08-29T14:02:27Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• Neural Voting Field for Camera-Space 3D Hand Pose Estimation [106.34750803910714]
  We present a unified framework for camera-space 3D hand pose estimation from a single RGB image based on 3D implicit representation. We propose a novel unified 3D dense regression scheme to estimate camera-space 3D hand pose via dense 3D point-wise voting in camera frustum.
  arXiv  Detail & Related papers  (2023-05-07T16:51:34Z)
• CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations [25.05308239278207]
  We present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity. We also introduce a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators.
  arXiv  Detail & Related papers  (2023-01-08T05:07:41Z)
• Stochastic Modeling for Learnable Human Pose Triangulation [0.7646713951724009]
  We propose a modeling framework for 3D human pose triangulation and evaluate its performance across different datasets and spatial camera arrangements. The proposed pose triangulation model successfully generalizes to different camera arrangements and between two public datasets.
  arXiv  Detail & Related papers  (2021-10-01T09:26:25Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild [98.69191256693703]
  We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.
  arXiv  Detail & Related papers  (2021-08-05T08:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.