MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects

• URL: http://arxiv.org/abs/2603.22839v1
• Date: Tue, 24 Mar 2026 06:26:28 GMT
• Title: MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects
• Abstract summary: Multi-camera dynamic Augmented Reality (AR) applications require a camera pose estimation to leverage individual information from each camera in one common system.<n>While commonly cameras are calibrated in an initial step or updated through the constant use of markers, another option is to leverage information already present in the scene, like known objects.<n>We propose a constant dynamic camera pose estimation leveraging FotemporalV overlaps of known objects on the fly.
• Score: 19.26912938809402
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-camera dynamic Augmented Reality (AR) applications require a camera pose estimation to leverage individual information from each camera in one common system. This can be achieved by combining contextual information, such as markers or objects, across multiple views. While commonly cameras are calibrated in an initial step or updated through the constant use of markers, another option is to leverage information already present in the scene, like known objects. Another downside of marker-based tracking is that markers have to be tracked inside the field-of-view (FoV) of the cameras. To overcome these limitations, we propose a constant dynamic camera pose estimation leveraging spatiotemporal FoV overlaps of known objects on the fly. To achieve that, we enhance the state-of-the-art object pose estimator to update our spatiotemporal scene graph, enabling a relation even among non-overlapping FoV cameras. To evaluate our approach, we introduce a multi-camera, multi-object pose estimation dataset with temporal FoV overlap, including static and dynamic cameras. Furthermore, in FoV overlapping scenarios, we outperform the state-of-the-art on the widely used YCB-V and T-LESS dataset in camera pose accuracy. Our performance on both previous and our proposed datasets validates the effectiveness of our marker-less approach for AR applications. The code and dataset are available on https://github.com/roth-hex-lab/IEEE-VR-2026-MultiCam.

Related papers

• Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos [14.791378262503967]
  We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple cameras.<n>We propose a two-stage optimization framework that decouples the task robust camera tracking and dense depth.<n>Our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2026-03-12T15:31:32Z)
• Multi-View 3D Point Tracking [67.21282192436031]
  We introduce the first data-driven multi-view 3D point tracker, designed to track arbitrary points in dynamic scenes using multiple camera views.<n>Our model directly predicts 3D correspondences using a practical number of cameras.<n>We train on 5K synthetic multi-view Kubric sequences and evaluate on two real-world benchmarks.
  arXiv  Detail & Related papers  (2025-08-28T17:58:20Z)
• FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video [52.33896173943054]
  Egocentric motion capture with a head-mounted body-facing stereo camera is crucial for VR and AR applications.<n>Existing methods rely on synthetic pretraining and struggle to generate smooth and accurate predictions in real-world settings.<n>We propose FRAME, a simple yet effective architecture that combines device pose and camera feeds for state-of-the-art body pose prediction.
  arXiv  Detail & Related papers  (2025-03-29T14:26:06Z)
• Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera [19.204896246140155]
  Event cameras possess remarkable attributes such as high dynamic range, low latency, and resilience against motion blur. We propose a line-based robust pose estimation and tracking method for planar or non-planar objects using an event camera.
  arXiv  Detail & Related papers  (2024-08-06T14:36:43Z)
• MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark [63.878793340338035]
  Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. Existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting. We present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments.
  arXiv  Detail & Related papers  (2024-03-29T15:08:37Z)
• VICAN: Very Efficient Calibration Algorithm for Large Camera Networks [49.17165360280794]
  We introduce a novel methodology that extends Pose Graph Optimization techniques. We consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step. Our framework retains compatibility with traditional PGO solvers, but its efficacy benefits from a custom-tailored optimization scheme.
  arXiv  Detail & Related papers  (2024-03-25T17:47:03Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation [101.55622133406446]
  We propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets.
  arXiv  Detail & Related papers  (2022-04-07T17:58:47Z)
• Simultaneous Multi-View Camera Pose Estimation and Object Tracking with Square Planar Markers [0.0]
  This work proposes a novel method to simultaneously solve the above-mentioned problems. From a video sequence showing a rigid set of planar markers recorded from multiple cameras, the proposed method is able to automatically obtain the three-dimensional configuration of the markers. Once the parameters are obtained, tracking of the object can be done in real time with a low computational cost.
  arXiv  Detail & Related papers  (2021-03-16T15:33:58Z)
• YCB-M: A Multi-Camera RGB-D Dataset for Object Recognition and 6DoF Pose Estimation [2.9972063833424216]
  We present a dataset of 32 scenes that have been captured by 7 different 3D cameras, totaling 49,294 frames. This allows evaluating the sensitivity of pose estimation algorithms to the specifics of the used camera.
  arXiv  Detail & Related papers  (2020-04-24T11:14:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.