CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras

• URL: http://arxiv.org/abs/2405.06845v1
• Date: Fri, 10 May 2024 23:02:23 GMT
• Title: CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras
• Authors: James Tang, Shashwat Suri, Daniel Ajisafe, Bastian Wandt, Helge Rhodin,
• Abstract summary: It is now possible to estimate 3D human pose from monocular images with off-the-shelf 3D pose estimators. Many practical applications require fine-grained absolute pose information for which multi-view cues and camera calibration are necessary. Our goal is full automation, which includes temporal synchronization, as well as intrinsic and extrinsic camera calibration.
• Score: 18.51320244029833
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is now possible to estimate 3D human pose from monocular images with off-the-shelf 3D pose estimators. However, many practical applications require fine-grained absolute pose information for which multi-view cues and camera calibration are necessary. Such multi-view recordings are laborious because they require manual calibration, and are expensive when using dedicated hardware. Our goal is full automation, which includes temporal synchronization, as well as intrinsic and extrinsic camera calibration. This is done by using persons in the scene as the calibration objects. Existing methods either address only synchronization or calibration, assume one of the former as input, or have significant limitations. A common limitation is that they only consider single persons, which eases correspondence finding. We attain this generality by partitioning the high-dimensional time and calibration space into a cascade of subspaces and introduce tailored algorithms to optimize each efficiently and robustly. The outcome is an easy-to-use, flexible, and robust motion capture toolbox that we release to enable scientific applications, which we demonstrate on diverse multi-view benchmarks. Project website: https://github.com/jamestang1998/CasCalib.

Related papers

• EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration [49.90228618894857]
  We introduce a new approach to hand-eye calibration called EasyHeC, which is markerless, white-box, and delivers superior accuracy and robustness. We propose to use two key technologies: differentiable rendering-based camera pose optimization and consistency-based joint space exploration. Our evaluation demonstrates superior performance in synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-02T03:49:54Z)
• SceneCalib: Automatic Targetless Calibration of Cameras and Lidars in Autonomous Driving [10.517099201352414]
  SceneCalib is a novel method for simultaneous self-calibration of extrinsic and intrinsic parameters in a system containing multiple cameras and a lidar sensor. We resolve issues with a fully automatic method that requires no explicit correspondences between camera images and lidar point clouds.
  arXiv  Detail & Related papers  (2023-04-11T23:02:16Z)
• Deep Learning for Camera Calibration and Beyond: A Survey [100.75060862015945]
  Camera calibration involves estimating camera parameters to infer geometric features from captured sequences. Recent efforts show that learning-based solutions have the potential to be used in place of the repeatability works of manual calibrations.
  arXiv  Detail & Related papers  (2023-03-19T04:00:05Z)
• Scene-Aware 3D Multi-Human Motion Capture from a Single Camera [83.06768487435818]
  We consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera. We leverage recent advances in computer vision using large-scale pre-trained models for a variety of modalities, including 2D body joints, joint angles, normalized disparity maps, and human segmentation masks. In particular, we estimate the scene depth and unique person scale from normalized disparity predictions using the 2D body joints and joint angles.
  arXiv  Detail & Related papers  (2023-01-12T18:01:28Z)
• Online Marker-free Extrinsic Camera Calibration using Person Keypoint Detections [25.393382192511716]
  We propose a marker-free online method for the extrinsic calibration of multiple smart edge sensors. Our method assumes the intrinsic camera parameters to be known and requires priming with a rough initial estimate of the camera poses. We show that the calibration with our method achieves lower reprojection errors compared to a reference calibration generated by an offline method.
  arXiv  Detail & Related papers  (2022-09-15T15:54:21Z)
• A Deep Perceptual Measure for Lens and Camera Calibration [35.03926427249506]
  In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters directly from a single image. We train this network using automatically generated samples from a large-scale panorama dataset. We conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters.
  arXiv  Detail & Related papers  (2022-08-25T18:40:45Z)
• MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision [72.5863451123577]
  We show how to train a neural model that can perform accurate 3D pose and camera estimation. Our method outperforms both classical bundle adjustment and weakly-supervised monocular 3D baselines.
  arXiv  Detail & Related papers  (2021-08-10T18:39:56Z)
• Dynamic Event Camera Calibration [27.852239869987947]
  We present the first dynamic event camera calibration algorithm. It calibrates directly from events captured during relative motion between camera and calibration pattern. As demonstrated through our results, the obtained calibration method is highly convenient and reliably calibrates from data sequences spanning less than 10 seconds.
  arXiv  Detail & Related papers  (2021-07-14T14:52:58Z)
• Wide-Baseline Multi-Camera Calibration using Person Re-Identification [27.965850489928457]
  We address the problem of estimating the 3D pose of a network of cameras for large-environment wide-baseline scenarios. Treating people in the scene as "keypoints" and associating them across different camera views can be an alternative method for obtaining correspondences. Our method first employs a re-ID method to associate human bounding boxes across cameras, then converts bounding box correspondences to point correspondences.
  arXiv  Detail & Related papers  (2021-04-17T15:09:18Z)
• Calibrated and Partially Calibrated Semi-Generalized Homographies [65.29477277713205]
  We propose the first minimal solutions for estimating the semi-generalized homography given a perspective and a generalized camera. The proposed solvers are stable and efficient as demonstrated by a number of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2021-03-11T08:56:24Z)
• Infrastructure-based Multi-Camera Calibration using Radial Projections [117.22654577367246]
  Pattern-based calibration techniques can be used to calibrate the intrinsics of the cameras individually. Infrastucture-based calibration techniques are able to estimate the extrinsics using 3D maps pre-built via SLAM or Structure-from-Motion. We propose to fully calibrate a multi-camera system from scratch using an infrastructure-based approach.
  arXiv  Detail & Related papers  (2020-07-30T09:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.