Spatiotemporal Multi-Camera Calibration using Freely Moving People

• URL: http://arxiv.org/abs/2502.12546v1
• Date: Tue, 18 Feb 2025 05:15:52 GMT
• Title: Spatiotemporal Multi-Camera Calibration using Freely Moving People
• Authors: Sang-Eun Lee, Ko Nishino, Shohei Nobuhara,
• Abstract summary: We propose a novel method for multi-camera calibration using freely moving people in multiview videos. We use 3D human poses obtained from an off-the-temporal monotemporal shelf to transform them into 3D points on a unit sphere. We employ a probabilistic approach that can jointly solve both problems of aligningtemporal data and establishing correspondences.
• Score: 32.288669810272864
• License:
• Abstract: We propose a novel method for spatiotemporal multi-camera calibration using freely moving people in multiview videos. Since calibrating multiple cameras and finding matches across their views are inherently interdependent, performing both in a unified framework poses a significant challenge. We address these issues as a single registration problem of matching two sets of 3D points, leveraging human motion in dynamic multi-person scenes. To this end, we utilize 3D human poses obtained from an off-the-shelf monocular 3D human pose estimator and transform them into 3D points on a unit sphere, to solve the rotation, time offset, and the association alternatingly. We employ a probabilistic approach that can jointly solve both problems of aligning spatiotemporal data and establishing correspondences through soft assignment between two views. The translation is determined by applying coplanarity constraints. The pairwise registration results are integrated into a multiview setup, and then a nonlinear optimization method is used to improve the accuracy of the camera poses, temporal offsets, and multi-person associations. Extensive experiments on synthetic and real data demonstrate the effectiveness and flexibility of the proposed method as a practical marker-free calibration tool.

Related papers

• Neural Real-Time Recalibration for Infrared Multi-Camera Systems [2.249916681499244]
  There are no learning-free or neural techniques for real-time recalibration of infrared multi-camera systems. We propose a neural network-based method capable of dynamic real-time calibration.
  arXiv  Detail & Related papers  (2024-10-18T14:37:37Z)
• CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras [18.51320244029833]
  It is now possible to estimate 3D human pose from monocular images with off-the-shelf 3D pose estimators. Many practical applications require fine-grained absolute pose information for which multi-view cues and camera calibration are necessary. Our goal is full automation, which includes temporal synchronization, as well as intrinsic and extrinsic camera calibration.
  arXiv  Detail & Related papers  (2024-05-10T23:02:23Z)
• Scene-Aware 3D Multi-Human Motion Capture from a Single Camera [83.06768487435818]
  We consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera. We leverage recent advances in computer vision using large-scale pre-trained models for a variety of modalities, including 2D body joints, joint angles, normalized disparity maps, and human segmentation masks. In particular, we estimate the scene depth and unique person scale from normalized disparity predictions using the 2D body joints and joint angles.
  arXiv  Detail & Related papers  (2023-01-12T18:01:28Z)
• Progressive Multi-view Human Mesh Recovery with Self-Supervision [68.60019434498703]
  Existing solutions typically suffer from poor generalization performance to new settings. We propose a novel simulation-based training pipeline for multi-view human mesh recovery.
  arXiv  Detail & Related papers  (2022-12-10T06:28:29Z)
• Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras [35.67288909201899]
  We propose a lightweight total motion capture system for multi-person interactive scenarios using only sparse multi-view cameras. Our method is capable of efficient localization and accurate association of the hands and faces even on severe occluded occasions. Overall, we propose the first light-weight total capture system and achieves fast, robust and accurate multi-person total motion capture performance.
  arXiv  Detail & Related papers  (2021-08-23T19:23:35Z)
• Calibrated and Partially Calibrated Semi-Generalized Homographies [65.29477277713205]
  We propose the first minimal solutions for estimating the semi-generalized homography given a perspective and a generalized camera. The proposed solvers are stable and efficient as demonstrated by a number of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2021-03-11T08:56:24Z)
• Iterative Greedy Matching for 3D Human Pose Tracking from Multiple Views [22.86745487695168]
  We propose an approach for estimating 3D human poses of multiple people from a set of calibrated cameras. Our approach builds upon a real-time 2D multi-person pose estimation system and greedily solves the association problem between multiple views.
  arXiv  Detail & Related papers  (2021-01-24T16:28:10Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)
• Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild [49.672487902268706]
  We present a framework that jointly estimates camera temporal alignment and 3D point triangulation. We reconstruct 3D motion trajectories of human bodies in events captured by multiple unsynchronized and unsynchronized video cameras.
  arXiv  Detail & Related papers  (2020-07-24T23:50:46Z)
• Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction [118.21363599332493]
  We present a method to leverage photometric consistency across time when annotations are only available for a sparse subset of frames in a video. Our model is trained end-to-end on color images to jointly reconstruct hands and objects in 3D by inferring their poses. We achieve state-of-the-art results on 3D hand-object reconstruction benchmarks and demonstrate that our approach allows us to improve the pose estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-28T12:03:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.