Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization

• URL: http://arxiv.org/abs/2604.01030v1
• Date: Wed, 01 Apr 2026 15:40:20 GMT
• Title: Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization
• Abstract summary: Diff3R is a novel framework that bridges feed-forward prediction and test-time optimization.<n>We show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods.
• Score: 76.38917994186733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in 3D Gaussian Splatting (3DGS) present two main directions: feed-forward models offer fast inference in sparse-view settings, while per-scene optimization yields high-quality renderings but is computationally expensive. To combine the benefits of both, we introduce Diff3R, a novel framework that explicitly bridges feed-forward prediction and test-time optimization. By incorporating a differentiable 3DGS optimization layer directly into the training loop, our network learns to predict an optimal initialization for test-time optimization rather than a conventional zero-shot result. To overcome the computational cost of backpropagating through the optimization steps, we propose computing gradients via the Implicit Function Theorem and a scalable, matrix-free PCG solver tailored for 3DGS optimization. Additionally, we incorporate a data-driven uncertainty model into the optimization process by adaptively controlling how much the parameters are allowed to change during optimization. This approach effectively mitigates overfitting in under-constrained regions and increases robustness against input outliers. Since our proposed optimization layer is model-agnostic, we show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods, providing improvements for test-time optimization.

Related papers

• ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting [65.02700555625749]
  ForeSplat is an optimization-aware training framework for feed-forward 3DGS models.<n>ForeSplat substantially reduces the capacity pressure on the feed-forward model.<n>It bridges the gap between amortized prediction and per-scene optimization.
  arXiv  Detail & Related papers  (2026-05-21T05:38:47Z)
• Learn2Splat: Extending the Horizon of Learned 3DGS Optimization [53.913063313338874]
  We introduce a learned for 3DGS that avoids degradation over extended horizons without auxiliary mechanisms.<n>Results improved novel view quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings.
  arXiv  Detail & Related papers  (2026-05-15T09:21:29Z)
• Faster-GS: Analyzing and Improving Gaussian Splatting Optimization [1.2949520455740091]
  We consolidate and evaluate the most effective and broadly applicable strategies from prior 3DGS research.<n> Faster-GS provides a rigorously optimized algorithm that we evaluate across a comprehensive suite of benchmarks.<n>Our experiments demonstrate that Faster-GS achieves up to 5$times$ faster training while maintaining visual quality.
  arXiv  Detail & Related papers  (2026-02-10T17:22:59Z)
• A Step to Decouple Optimization in 3DGS [38.797134528503015]
  3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time novel view synthesis.<n>In this paper, we take a step to decouple the process into: Sparse Adam, Re-State Regularization and Decoupled Attribute Regularization.
  arXiv  Detail & Related papers  (2026-01-23T13:34:39Z)
• Opt3DGS: Optimizing 3D Gaussian Splatting with Adaptive Exploration and Curvature-Aware Exploitation [10.150288678666001]
  3D Splatting (3DGS) has emerged as a leading framework for novel view synthesis, yet its core optimization challenges remain underexplored.<n>We identify two key issues in 3DGS optimization: entrapment in suboptimal local optima and insufficient convergence quality.<n>We propose Opt3DGS, a robust framework that enhances 3DGS through a two-stage optimization process.
  arXiv  Detail & Related papers  (2025-11-17T16:37:33Z)
• DashGaussian: Optimizing 3D Gaussian Splatting in 200 Seconds [71.37326848614133]
  We propose DashGaussian, a scheduling scheme over the optimization complexity of 3DGS.<n>We show that our method accelerates the optimization of various 3DGS backbones by 45.7% on average.
  arXiv  Detail & Related papers  (2025-03-24T07:17:27Z)
• Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator [75.05106948314956]
  We show that an increasing large momentum parameter for the first-order moment is sufficient for adaptive scaling.<n>We also give insights for increasing the momentum in a stagewise manner in accordance with stagewise decreasing step size.
  arXiv  Detail & Related papers  (2021-04-30T08:50:24Z)
• A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning [95.85269649177336]
  ZO optimization iteratively performs three major steps: gradient estimation, descent direction, and solution update. We demonstrate promising applications of ZO optimization, such as evaluating and generating explanations from black-box deep learning models, and efficient online sensor management.
  arXiv  Detail & Related papers  (2020-06-11T06:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.