ORBIT++: Benchmarking SfM in the Wild with 360° Video
Abstract Overview
This paper presents ORBIT++, an updated version of the ORBIT benchmark for evaluating structure-from-motion and camera pose estimation on difficult real-world videos derived from online 360° footage. The benchmark is built by estimating pseudo-ground-truth trajectories from full panoramic videos using a custom rig-based SfM pipeline, then rendering challenging perspective clips with varied fields of view and rotation patterns. After filtering and validation, the dataset contains 308 clips drawn from 77 unique 360° videos, covering scenarios such as low texture, dark scenes, high-speed motion, crowds, egocentric captures, and fluid motion. The authors position the benchmark as a way to test methods in settings where existing synthetic or simpler datasets do not adequately reflect in-the-wild failure cases.
Novelty
The distinctive contribution is leveraging panoramic 360° web videos to recover reliable camera trajectories across a full omnidirectional field of view and then deriving difficult perspective-view benchmark clips from those same videos. This setup separates the omnidirectional information used to construct ground truth from the limited-view perspective inputs used during evaluation.
Results
Evaluations on ORBIT show that classical, optimization-based, and feed-forward methods all struggle on the benchmark, with every evaluated baseline exhibiting a failure rate above 36%. MegaSaM and RoMo+MegaSaM attain the lowest mean absolute trajectory errors (0.67 m and 0.59 m) while still failing on 39.09% and 38.56% of clips, whereas COLMAP fails on 56.49%, and ORB-SLAM2, VGGT-Long, and MonST3R fail on over 94% of clips under the evaluation threshold. The ground-truth generation pipeline achieves 0.07 ± 0.04 m ATE on the 360Loc dataset and 0.0 ± 0.0 m on Synthetic 360, confirming its high fidelity.
Key Points
- ORBIT++ constructs a challenging SfM benchmark from 360° web videos by estimating trajectories on the full panorama and then rendering difficult perspective clips with controlled rotations and fields of view.
- The benchmark contains 308 validated clips from 77 source videos and explicitly covers multiple real-world challenge categories, including low texture, dark lighting, high camera speeds, crowds, egocentric motion, and fluid scenes.
- Across evaluated baselines, no method is robust on ORBIT: even top-performing methods retain failure rates near 39%, with challenge analysis revealing distinct failure profiles between bundle-adjustment and feed-forward approaches.