SteerPose: Simultaneous Extrinsic Camera Calibration and Matching from Articulation

• URL: http://arxiv.org/abs/2506.01691v2
• Date: Thu, 07 Aug 2025 12:38:31 GMT
• Title: SteerPose: Simultaneous Extrinsic Camera Calibration and Matching from Articulation
• Authors: Sang-Eun Lee, Ko Nishino, Shohei Nobuhara,
• Abstract summary: We propose SteerPose, a neural network that performs a rotation of 2D poses into another view.<n>By integrating differentiable matching, SteerPose simultaneously performs extrinsic camera calibration and correspondence search.<n>We demonstrate that our method can reconstruct the 3D poses of novel animals in multi-camera setups.
• Score: 32.288669810272864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Can freely moving humans or animals themselves serve as calibration targets for multi-camera systems while simultaneously estimating their correspondences across views? We humans can solve this problem by mentally rotating the observed 2D poses and aligning them with those in the target views. Inspired by this cognitive ability, we propose SteerPose, a neural network that performs this rotation of 2D poses into another view. By integrating differentiable matching, SteerPose simultaneously performs extrinsic camera calibration and correspondence search within a single unified framework. We also introduce a novel geometric consistency loss that explicitly ensures that the estimated rotation and correspondences result in a valid translation estimation. Experimental results on diverse in-the-wild datasets of humans and animals validate the effectiveness and robustness of the proposed method. Furthermore, we demonstrate that our method can reconstruct the 3D poses of novel animals in multi-camera setups by leveraging off-the-shelf 2D pose estimators and our class-agnostic model.

Related papers

• Spatiotemporal Multi-Camera Calibration using Freely Moving People [32.288669810272864]
  We propose a novel method for multi-camera calibration using freely moving people in multiview videos.<n>We use 3D human poses obtained from an off-the-temporal monotemporal shelf to transform them into 3D points on a unit sphere.<n>We employ a probabilistic approach that can jointly solve both problems of aligningtemporal data and establishing correspondences.
  arXiv  Detail & Related papers  (2025-02-18T05:15:52Z)
• UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
  UPose3D is a novel approach for multi-view 3D human pose estimation. It improves robustness and flexibility without requiring direct 3D annotations.
  arXiv  Detail & Related papers  (2024-04-23T00:18:00Z)
• Unsupervised Multi-Person 3D Human Pose Estimation From 2D Poses Alone [4.648549457266638]
  We present one of the first studies investigating the feasibility of unsupervised multi-person 2D-3D pose estimation. Our method involves independently lifting each subject's 2D pose to 3D, before combining them in a shared 3D coordinate system. This by itself enables us to retrieve an accurate 3D reconstruction of their poses.
  arXiv  Detail & Related papers  (2023-09-26T11:42:56Z)
• RelPose++: Recovering 6D Poses from Sparse-view Observations [66.6922660401558]
  We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
  arXiv  Detail & Related papers  (2023-05-08T17:59:58Z)
• CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations [25.05308239278207]
  We present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity. We also introduce a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators.
  arXiv  Detail & Related papers  (2023-01-08T05:07:41Z)
• PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision [102.48681650013698]
  Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions to guide the learning. We propose a novel self-supervised approach that allows us to explicitly generate 2D-3D pose pairs for augmenting supervision. This is made possible via introducing a reinforcement-learning-based imitator, which is learned jointly with a pose estimator alongside a pose hallucinator.
  arXiv  Detail & Related papers  (2022-03-29T14:45:53Z)
• MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision [72.5863451123577]
  We show how to train a neural model that can perform accurate 3D pose and camera estimation. Our method outperforms both classical bundle adjustment and weakly-supervised monocular 3D baselines.
  arXiv  Detail & Related papers  (2021-08-10T18:39:56Z)
• Evaluation of deep lift pose models for 3D rodent pose estimation based on geometrically triangulated data [1.84316002191515]
  Behavior is typically studied in terms of pose changes, which are ideally captured in three dimensions. This requires triangulation over a multi-camera system which view the animal from different angles. Here we propose the usage of lift-pose models that allow for robust 3D pose estimation of freely moving rodents from a single view camera view.
  arXiv  Detail & Related papers  (2021-06-24T13:08:33Z)
• PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation [83.50127973254538]
  Existing 3D human pose estimators suffer poor generalization performance to new datasets. We present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity.
  arXiv  Detail & Related papers  (2021-05-06T06:57:42Z)
• View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose [36.384824115033304]
  We propose an approach to learning a compact view-invariant embedding space from 2D body joint keypoints, without explicitly predicting 3D poses. Experimental results show that our embedding model achieves higher accuracy when retrieving similar poses across different camera views.
  arXiv  Detail & Related papers  (2020-10-23T17:58:35Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.