BA-T: An Iterative Transformer for Two-View Bundle Adjustment

• URL: http://arxiv.org/abs/2606.03287v1
• Date: Tue, 02 Jun 2026 07:51:14 GMT
• Title: BA-T: An Iterative Transformer for Two-View Bundle Adjustment
• Abstract summary: We propose BA-T, an iterative Transformer that implements BA-style structured updates as a repeatable layer in implicit token space.<n> experiments demonstrate that BA-T progressively improves pose and reconstruction accuracy across iterations.<n>BA-T provides a compact, efficient, and structural alternative to depth-heavy attention, enabling accurate 3D reconstruction within a lightweight architecture.
• Score: 53.16990349592508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feed-forward models for 3D reconstruction have achieved strong performance using deep cross-view attention to exchange information across images. However, these approaches often depend on heavy decoder stacks and lack a structured mechanism for geometry refinement, resulting in poor multi-view consistency. We address this by drawing inspiration from classical bundle adjustment (BA), which can be viewed as an iterative information propagation process between poses and local geometry. Inspired by BA, we propose BA-T, an iterative Transformer that implements BA-style structured updates as a repeatable layer in implicit token space. Instead of relying on deep attention stacks, BA-T refines predictions based on latent residual by a single lightweight layer. Experiments demonstrate that BA-T progressively improves pose and reconstruction accuracy across iterations, achieves stronger cross-view consistency than conventional decoders, and matches or surpasses substantially larger models while using only 16% of their decoder parameters. BA-T provides a compact, efficient, and structural alternative to depth-heavy attention, enabling accurate 3D reconstruction within a lightweight architecture. The code will be made publicly at https://github.com/zhangganlin/BA-T.

Related papers

• CG-GLORE: A Conjugate Gradient-Based Global-Local Regularization Network for Sparse-View CT Reconstruction [3.7567584936415166]
  Sparse-view computed tomography (CT) reduces radiation dose by acquiring fewer projection views, but the resulting inverse problem is highly ill-posed.<n>Existing deep reconstruction methods have achieved promising performance, yet many rely on first-order updates or large regularization networks.<n>We propose textbfCG-GLORE, a compact deep unrolling framework inspired by second-order optimization for sparse-view CT reconstruction.
  arXiv  Detail & Related papers  (2026-08-15T14:08:44Z)
• Déjà View: Looping Transformers for Multi-View 3D Reconstruction [109.98721386523641]
  Recent feed-forward 3D reconstruction transformers have scaled to over a billion parameters.<n>We show that model depth partially buys iteration, paid for inefficiently in unique parameters, and instead make that iteration explicit in architecture.<n>Our model, DéjView, applies a single looped transformer block recurrently to per-view features for K refinement steps.
  arXiv  Detail & Related papers  (2026-05-28T16:47:48Z)
• TurboVGGT: Fast Visual Geometry Reconstruction with Adaptive Alternating Attention [21.29668311125256]
  TurboVGGT employs an efficient visual geometry transformer with adaptive alternating attention for fast multi-view 3D reconstruction.<n>In the adaptive sparse global attention, TurboVGGT adaptively learns representative tokens with varying sparsity levels for global geometry modeling.<n>Experiments on multiple 3D reconstruction benchmarks demonstrate that TurboVGGT achieves fast multi-view reconstruction while maintaining competitive reconstruction quality.
  arXiv  Detail & Related papers  (2026-05-14T03:24:09Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows [10.300202521638274]
  We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows impacts feed-forward 3D reconstruction.<n>We show that expanding the context window -- by substantially increasing the number of active object and image tokens -- remarkably narrows this gap and enables high-fidelity 3D object reconstruction and inverse rendering.
  arXiv  Detail & Related papers  (2026-04-06T21:21:12Z)
• Reliev3R: Relieving Feed-forward Reconstruction from Multi-View Geometric Annotations [98.66466590444553]
  We propose Reliev3R, a weakly-supervised paradigm for training FFRMs from scratch without cost-prohibitive multi-view geometric annotations.<n>At the core of Reliev3R, we design an ambiguity-aware relative depth loss and a trigonometry-based reprojection loss to facilitate supervision for multi-view geometric consistency.
  arXiv  Detail & Related papers  (2026-04-01T06:46:54Z)
• HeBA: Heterogeneous Bottleneck Adapters for Robust Vision-Language Models [3.8707695363745214]
  Heterogeneous Bottleneck Adapter (HeBA) is a unified architectural framework that introduces modality-specific inductive biases.<n>HeBA processes visual tokens via 2D depthwise-separable convolutions to preserve spatial correlations.<n>Unlike standard expanding adapters, HeBA employs a compression bottleneck that explicitly forces the model to learn compact, robust features.
  arXiv  Detail & Related papers  (2026-03-17T15:23:04Z)
• Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation [16.22677555146353]
  Fin3R is a simple, effective, and general fine-tuning method for feed-forward 3D reconstruction models.<n>We validate our method on a wide range of models, including DUSt3R, MASt3R, CUT3R, and VGGT.
  arXiv  Detail & Related papers  (2025-11-27T13:10:19Z)
• LARM: A Large Articulated-Object Reconstruction Model [29.66486888001511]
  LARM is a unified feedforward framework that reconstructs 3D articulated objects from sparse-view images.<n>LARM generates auxiliary outputs such as depth maps and part masks to facilitate explicit 3D mesh extraction and joint estimation.<n>Our pipeline eliminates the need for dense supervision and supports high-fidelity reconstruction across diverse object categories.
  arXiv  Detail & Related papers  (2025-11-14T18:55:27Z)
• Light-SQ: Structure-aware Shape Abstraction with Superquadrics for Generated Meshes [60.92139345612904]
  We present Light-SQ, a novel superquadric-based optimization framework.<n>We propose a block-regrow-fill strategy guided by structure-aware volumetric decomposition.<n>Experiments demonstrate that Light-SQ enables efficient, high-fidelity, and editable shape abstraction with superquadrics.
  arXiv  Detail & Related papers  (2025-09-29T16:18:32Z)
• STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer [72.88105562624838]
  We present STream3R, a novel approach to 3D reconstruction that reformulates pointmap prediction as a decoder-only Transformer problem.<n>By learning geometric priors from large-scale 3D datasets, STream3R generalizes well to diverse and challenging scenarios.<n>Our results underscore the potential of causal Transformer models for online 3D perception, paving the way for real-time 3D understanding in streaming environments.
  arXiv  Detail & Related papers  (2025-08-14T17:58:05Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.