Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction

• URL: http://arxiv.org/abs/2605.11354v1
• Date: Tue, 12 May 2026 00:20:02 GMT
• Title: Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction
• Abstract summary: Lite3R is a model-agnostic teacher-student framework that replaces dense attention with Sparse Linear Attention.<n>Lite3R substantially reduces latency (1.7-2.0x) and memory usage (1.9-2.4x) while preserving competitive reconstruction quality overall.
• Score: 23.223291360701907
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Transformer-based 3D reconstruction has emerged as a powerful paradigm for recovering geometry and appearance from multi-view observations, offering strong performance across challenging visual conditions. As these models scale to larger backbones and higher-resolution inputs, improving their efficiency becomes increasingly important for practical deployment. However, modern 3D transformer pipelines face two coupled challenges: dense multi-view attention creates substantial token-mixing overhead, and low-precision execution can destabilize geometry-sensitive representations and degrade depth, pose, and 3D consistency. To address the first challenge, we propose Lite3R, a model-agnostic teacher-student framework that replaces dense attention with Sparse Linear Attention to preserve important geometric interactions while reducing attention cost. To address the second challenge, we introduce a parameter-efficient FP8-aware quantization-aware training (FP8-aware QAT) strategy with partial attention distillation, which freezes the vast majority of pretrained backbone parameters and trains only lightweight linear-branch projection layers, enabling stable low-precision deployment while retaining pretrained geometric priors. We further evaluate Lite3R on two representative backbones, VGGT and DA3-Large, over BlendedMVS and DTU64, showing that it substantially reduces latency (1.7-2.0x) and memory usage (1.9-2.4x) while preserving competitive reconstruction quality overall. These results demonstrate that Lite3R provides an effective algorithm-system co-design approach for practical transformer-based 3D reconstruction. Code: https://github.com/AIGeeksGroup/Lite3R. Website: https://aigeeksgroup.github.io/Lite3R.

Related papers

• QuerySplat: Decoupling Geometry and Appearance Representations in 3DGS Prediction [25.05567017307621]
  We propose textbfQuerySplat, a feed-forward 3DGS framework driven by geometric priors and explicit appearance decoupling.<n>Experiments demonstrate that QuerySplat mitigates the blurry rendering issues of early query-based models.<n>On the challenging DL3DV benchmark, it achieves state-of-the-art novel view synthesis performance, with average PSNR gains of 2.30 dB and 1.04 dB over the best pose-free and pose-required baselines.
  arXiv  Detail & Related papers  (2026-08-02T12:12:02Z)
• Tri-Efficient Transfer Learning for Point Cloud Videos [32.14641034344552]
  PoinTriE aims to mine richer supervision signals from existing data rather than blindly scaling datasets.<n>For pre-training, pseudo-motion trajectories are synthesized via rigid transformations, paired with text corpora and 2D projections derived from raw point clouds.<n>During fine-tuning, we freeze the pretrained backbone and only update a lightweight Spatio-temporal Side Network built with LoRA units.
  arXiv  Detail & Related papers  (2026-06-23T05:55:02Z)
• PointSplat: Efficient Geometry-Driven Pruning and Transformer Refinement for 3D Gaussian Splatting [2.5823344978857388]
  3D Gaussian Splatting has recently unlocked real-time, high-fidelity novel view synthesis by representing scenes using explicit 3D primitives.<n>Traditional methods often require millions of Gaussians to capture complex scenes, leading to significant memory and storage demands.<n>Recent approaches have addressed this issue through pruning and per-scene fine-tuning of Gaussian parameters, thereby reducing the model size while maintaining visual quality.
  arXiv  Detail & Related papers  (2026-04-10T20:56:18Z)
• LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows [10.300202521638274]
  We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows impacts feed-forward 3D reconstruction.<n>We show that expanding the context window -- by substantially increasing the number of active object and image tokens -- remarkably narrows this gap and enables high-fidelity 3D object reconstruction and inverse rendering.
  arXiv  Detail & Related papers  (2026-04-06T21:21:12Z)
• MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation [59.75554954111619]
  We introduce Multi-view 3D Referring Expression (MV-3DRES), where the model must recover scene structure and segment the referred object directly from sparse multi-view images.<n>We propose the Multimodal Visual Geometry Grounded Transformer (MVGGT), an efficient end-to-end framework that integrates language information into sparse-view geometric reasoning.<n>Experiments show that MVGGT establishes the first strong baseline and achieves both high accuracy and fast inference, outperforming existing alternatives.
  arXiv  Detail & Related papers  (2026-01-11T11:44:07Z)
• LitePT: Lighter Yet Stronger Point Transformer [50.6430530112838]
  We analyse the role of different computational blocks in 3D point cloud networks.<n>We propose a new, improved 3D point cloud backbone that employs convolutions in early stages and switches to attention for deeper layers.<n>The resulting LitePT model has $3.6times$ fewer parameters, runs $2times$ faster, and uses $2times$ less memory than the state-of-the-art Point Transformer V3.
  arXiv  Detail & Related papers  (2025-12-15T18:59:57Z)
• HBSplat: Robust Sparse-View Gaussian Reconstruction with Hybrid-Loss Guided Depth and Bidirectional Warping [11.035994094874141]
  HBSplat is a framework that seamlessly integrates robust structural cues, virtual view constraints, and occluded region completion.<n> HBSplat sets a new state-of-the-art, achieving up to 21.13 dB PSNR and 0.189 LPIPS, while maintaining real-time inference.
  arXiv  Detail & Related papers  (2025-09-29T15:03:31Z)
• Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation [42.23472421492995]
  We show how to redesign the framework based on the characteristics of high-dimensional 3D images.<n>Our approach, VeloxSeg, begins with a deployable and dual-stream CNN-Transformer architecture.<n>VeloxSeg achieves a 26% Dice improvement, alongside increasing GPU throughput by 11x and CPU by 48x.
  arXiv  Detail & Related papers  (2025-09-26T13:12:43Z)
• STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer [72.88105562624838]
  We present STream3R, a novel approach to 3D reconstruction that reformulates pointmap prediction as a decoder-only Transformer problem.<n>By learning geometric priors from large-scale 3D datasets, STream3R generalizes well to diverse and challenging scenarios.<n>Our results underscore the potential of causal Transformer models for online 3D perception, paving the way for real-time 3D understanding in streaming environments.
  arXiv  Detail & Related papers  (2025-08-14T17:58:05Z)
• On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation [52.96632954620623]
  We introduce a novel geometry-aware PEFT module specifically designed for 3D point cloud transformers.<n>Our approach sets a new benchmark for efficient, scalable, and geometry-aware fine-tuning of large-scale 3D point cloud models.
  arXiv  Detail & Related papers  (2025-05-28T15:08:36Z)
• Rep3D: Re-parameterize Large 3D Kernels with Low-Rank Receptive Modeling for Medical Imaging [15.142146104837005]
  Rep3D is a 3D convolutional framework that incorporates a learnable spatial volumetric prior into large kernel training.<n>Rep3D offers an interpretable, and scalable solution for 3D medical image analysis.
  arXiv  Detail & Related papers  (2025-05-26T07:12:56Z)
• Multi-initialization Optimization Network for Accurate 3D Human Pose and Shape Estimation [75.44912541912252]
  We propose a three-stage framework named Multi-Initialization Optimization Network (MION) In the first stage, we strategically select different coarse 3D reconstruction candidates which are compatible with the 2D keypoints of input sample. In the second stage, we design a mesh refinement transformer (MRT) to respectively refine each coarse reconstruction result via a self-attention mechanism. Finally, a Consistency Estimation Network (CEN) is proposed to find the best result from mutiple candidates by evaluating if the visual evidence in RGB image matches a given 3D reconstruction.
  arXiv  Detail & Related papers  (2021-12-24T02:43:58Z)
• A New Backbone for Hyperspectral Image Reconstruction [90.48427561874402]
  3D hyperspectral image (HSI) reconstruction refers to inverse process of snapshot compressive imaging. Proposal is for a Spatial/Spectral Invariant Residual U-Net, namely SSI-ResU-Net. We show that SSI-ResU-Net achieves competing performance with over 77.3% reduction in terms of floating-point operations.
  arXiv  Detail & Related papers  (2021-08-17T16:20:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.