MirrorCalib: Utilizing Human Pose Information for Mirror-based Virtual Camera Calibration

• URL: http://arxiv.org/abs/2311.02791v3
• Date: Fri, 17 May 2024 22:45:22 GMT
• Title: MirrorCalib: Utilizing Human Pose Information for Mirror-based Virtual Camera Calibration
• Authors: Longyun Liao, Rong Zheng, Andrew Mitchell,
• Abstract summary: We present the novel task of estimating the parameters of a virtual camera relative to a real camera in exercise videos with a mirror. Prior knowledge of a human body and 2D joint locations are utilized to estimate the camera extrinsic parameters. MirrorCalib achieves a rotation error of 1.82deg and a translation error of 69.51 mm on a collected real-world dataset.
• Score: 3.776930498297967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present the novel task of estimating the extrinsic parameters of a virtual camera relative to a real camera in exercise videos with a mirror. This task poses a significant challenge in scenarios where the views from the real and mirrored cameras have no overlap or share salient features. To address this issue, prior knowledge of a human body and 2D joint locations are utilized to estimate the camera extrinsic parameters when a person is in front of a mirror. We devise a modified eight-point algorithm to obtain an initial estimation from 2D joint locations. The 2D joint locations are then refined subject to human body constraints. Finally, a RANSAC algorithm is employed to remove outliers by comparing their epipolar distances to a predetermined threshold. MirrorCalib achieves a rotation error of 1.82{\deg} and a translation error of 69.51 mm on a collected real-world dataset, which outperforms the state-of-art method.

Related papers

• SRPose: Two-view Relative Pose Estimation with Sparse Keypoints [51.49105161103385]
  SRPose is a sparse keypoint-based framework for two-view relative pose estimation in camera-to-world and object-to-camera scenarios. It achieves competitive or superior performance compared to state-of-the-art methods in terms of accuracy and speed. It is robust to different image sizes and camera intrinsics, and can be deployed with low computing resources.
  arXiv  Detail & Related papers  (2024-07-11T05:46:35Z)
• RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [72.13154206106259]
  We propose a novel pipeline that decouples the 6D pose and size estimation to mitigate the influence of imperfect scales on rigid transformations. Specifically, we leverage a pre-trained monocular estimator to extract local geometric information. A separate branch is designed to directly recover the metric scale of the object based on category-level statistics.
  arXiv  Detail & Related papers  (2023-09-19T02:20:26Z)
• RelPose++: Recovering 6D Poses from Sparse-view Observations [66.6922660401558]
  We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
  arXiv  Detail & Related papers  (2023-05-08T17:59:58Z)
• Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction [66.10717041384625]
  Zolly is the first 3DHMR method focusing on perspective-distorted images. We propose a new camera model and a novel 2D representation, termed distortion image, which describes the 2D dense distortion scale of the human body. We extend two real-world datasets tailored for this task, all containing perspective-distorted human images.
  arXiv  Detail & Related papers  (2023-03-24T04:22:41Z)
• Multi-task Learning for Camera Calibration [3.274290296343038]
  We present a unique method for predicting intrinsic (principal point offset and focal length) and extrinsic (baseline, pitch, and translation) properties from a pair of images. By reconstructing the 3D points using a camera model neural network and then using the loss in reconstruction to obtain the camera specifications, this innovative camera projection loss (CPL) method allows us that the desired parameters should be estimated.
  arXiv  Detail & Related papers  (2022-11-22T17:39:31Z)
• Ambiguity-Aware Multi-Object Pose Optimization for Visually-Assisted Robot Manipulation [17.440729138126162]
  We present an ambiguity-aware 6D object pose estimation network, PrimA6D++, as a generic uncertainty prediction method. The proposed method shows a significant performance improvement in T-LESS and YCB-Video datasets. We further demonstrate real-time scene recognition capability for visually-assisted robot manipulation.
  arXiv  Detail & Related papers  (2022-11-02T08:57:20Z)
• 3D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projections [2.486571221735935]
  We propose to directly optimise for localisation in 3D by training 2D CNNs end-to-end based on a 3D loss. Using videos from the MVOR dataset, we show that this end-to-end approach outperforms optimisation in 2D space.
  arXiv  Detail & Related papers  (2022-10-21T09:00:02Z)
• Self-Supervised Camera Self-Calibration from Video [34.35533943247917]
  We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models. Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods.
  arXiv  Detail & Related papers  (2021-12-06T19:42:05Z)
• DeepI2P: Image-to-Point Cloud Registration via Deep Classification [71.3121124994105]
  DeepI2P is a novel approach for cross-modality registration between an image and a point cloud. Our method estimates the relative rigid transformation between the coordinate frames of the camera and Lidar. We circumvent the difficulty by converting the registration problem into a classification and inverse camera projection optimization problem.
  arXiv  Detail & Related papers  (2021-04-08T04:27:32Z)
• Structure of Multiple Mirror System from Kaleidoscopic Projections of Single 3D Point [14.345346642066511]
  This paper proposes a novel algorithm of discovering the structure of a kaleidoscopic imaging system that consists of multiple planar mirrors and a camera. The key contribution of this paper is to propose novel algorithms for these problems using a single 3D point of unknown geometry.
  arXiv  Detail & Related papers  (2021-03-29T11:12:15Z)
• CosyPose: Consistent multi-view multi-object 6D pose estimation [48.097599674329004]
  We present a single-view single-object 6D pose estimation method, which we use to generate 6D object pose hypotheses. Second, we develop a robust method for matching individual 6D object pose hypotheses across different input images. Third, we develop a method for global scene refinement given multiple object hypotheses and their correspondences across views.
  arXiv  Detail & Related papers  (2020-08-19T14:11:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.