Learn2Splat: Extending the Horizon of Learned 3DGS Optimization

• URL: http://arxiv.org/abs/2605.15760v1
• Date: Fri, 15 May 2026 09:21:29 GMT
• Title: Learn2Splat: Extending the Horizon of Learned 3DGS Optimization
• Abstract summary: We introduce a learned for 3DGS that avoids degradation over extended horizons without auxiliary mechanisms.<n>Results improved novel view quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings.
• Score: 53.913063313338874
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: 3D Gaussian Splatting (3DGS) optimization is most commonly performed using standard optimizers (Adam, SGD). While stable across diverse scenes, standard optimizers are general-purpose and not tailored to the structure of the problem. In particular, they produce independent parameter updates that do not capture the structural and spatial relationships within a scene, leading to inefficient optimization and slow convergence. Recent works introduced learned optimizers that predict correlated updates informed by inter-parameter and inter-Gaussian dependencies. However, these methods are trained for a fixed number of optimization iterations and rely on manually scheduled learning rates to avoid degradation. In this paper, we introduce a learned optimizer for 3DGS that avoids degradation over extended optimization horizons without auxiliary mechanisms. To enable this, we propose a meta-learning scheme that extends the optimization horizon via a checkpoint buffer and an optimizer rollout strategy, combined with an architecture that encodes gradient scale information in its latent states. Results show improved early novel view synthesis quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings. To support our findings, we introduce the first unified framework for training and evaluating both learned and conventional optimizers across sparse and dense view settings. Code and models will be released publicly. Our project page is available at https://naamapearl.github.io/learn2splat .

Related papers

• ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting [65.02700555625749]
  ForeSplat is an optimization-aware training framework for feed-forward 3DGS models.<n>ForeSplat substantially reduces the capacity pressure on the feed-forward model.<n>It bridges the gap between amortized prediction and per-scene optimization.
  arXiv  Detail & Related papers  (2026-05-21T05:38:47Z)
• Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations [98.44542103979735]
  First-order gradient descent methods, epitomized by gradient descent (SGD) and Adam, serve as the cornerstone of modern training pipelines.<n>Large-scale model training, stringent privacy requirements, and distributed learning paradigms expose critical limitations in these conventional approaches regarding privacy protection and memory efficiency.<n>We retrospectively analyze the evolutionary trajectory of deep learning optimization algorithms and present a comprehensive empirical evaluation of mainstreams across diverse model architectures and training scenarios.<n>We distill key emerging trends and fundamental design trade-offs, pinpointing promising directions for future research.
  arXiv  Detail & Related papers  (2026-04-14T17:01:36Z)
• Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization [76.38917994186733]
  Diff3R is a novel framework that bridges feed-forward prediction and test-time optimization.<n>We show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods.
  arXiv  Detail & Related papers  (2026-04-01T15:40:20Z)
• Faster-GS: Analyzing and Improving Gaussian Splatting Optimization [1.2949520455740091]
  We consolidate and evaluate the most effective and broadly applicable strategies from prior 3DGS research.<n> Faster-GS provides a rigorously optimized algorithm that we evaluate across a comprehensive suite of benchmarks.<n>Our experiments demonstrate that Faster-GS achieves up to 5$times$ faster training while maintaining visual quality.
  arXiv  Detail & Related papers  (2026-02-10T17:22:59Z)
• A Step to Decouple Optimization in 3DGS [38.797134528503015]
  3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time novel view synthesis.<n>In this paper, we take a step to decouple the process into: Sparse Adam, Re-State Regularization and Decoupled Attribute Regularization.
  arXiv  Detail & Related papers  (2026-01-23T13:34:39Z)
• QuickSplat: Fast 3D Surface Reconstruction via Learned Gaussian Initialization [69.50126552763157]
  Surface reconstruction is fundamental to computer vision and graphics, enabling applications in 3D modeling, mixed reality, robotics, and more.<n>Existing approaches based on rendering obtain promising results, but optimize on a per-scene basis, resulting in a slow optimization that can struggle to model textureless regions.<n>We introduce QuickSplat, which learns data-driven priors to generate dense initializations for 2D gaussian splatting optimization of large-scale indoor scenes.
  arXiv  Detail & Related papers  (2025-05-08T18:43:26Z)
• Narrowing the Focus: Learned Optimizers for Pretrained Models [24.685918556547055]
  We propose a novel technique that learns a layer-specific linear combination of update directions provided by a set of base work tasks. When evaluated on an image, this specialized significantly outperforms both traditional off-the-shelf methods such as Adam, as well existing general learneds.
  arXiv  Detail & Related papers  (2024-08-17T23:55:19Z)
• A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning [95.85269649177336]
  ZO optimization iteratively performs three major steps: gradient estimation, descent direction, and solution update. We demonstrate promising applications of ZO optimization, such as evaluating and generating explanations from black-box deep learning models, and efficient online sensor management.
  arXiv  Detail & Related papers  (2020-06-11T06:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.