DualDiff3D: Dual Structure-Appearance Diffusion Priors for Reliability-Enhanced 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2609.01516v1
• Date: Tue, 01 Sep 2026 16:45:53 GMT
• Title: DualDiff3D: Dual Structure-Appearance Diffusion Priors for Reliability-Enhanced 3D Gaussian Splatting
• Abstract summary: 3D reconstruction and novel-view synthesis often lead to poor reconstruction quality and artifacts in rendered novel views.<n>Recent efforts attempt to utilize powerful diffusion priors, yet they typically process rendered and reference views along an additional dimension in a single network.<n>We propose DualDiff, a novel pipeline that leverages dual diffusion priors with a Structure-Appearance Attention (SAA) module.<n>Our approach outperforms state-of-the-art methods even in the inference-only setting, with further performance gains achievable through training.
• Score: 71.316050612276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction and novel-view synthesis, scenarios with limited input views often lead to poor reconstruction quality and artifacts in rendered novel views. Recent efforts attempt to utilize powerful diffusion priors, yet they typically process rendered and reference views concatenated along an additional dimension in a single network. These methods overlook an inherent nature that different views should maintain appearance similarity but differ in structure due to view shifts, leading to blur caused by conflicts between the two properties. In this paper, we propose DualDiff, a novel pipeline that leverages dual diffusion priors with a Structure-Appearance Attention (SAA) module to introduce reference guidance for refining low-quality novel views rendered from flawed 3D representations. Specifically, we retain one diffusion branch to focus on extracting structural information from the low-quality novel views, while introducing another branch to ensure appearance consistency with reference views. Furthermore, we present a 3D reconstruction framework named DualDiff3D, which integrates a reliability-enhanced Render-Refine-Optimize (RRO) loop to progressively and robustly incorporate the refined novel views, yielding more accurate 3DGS. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods even in the inference-only setting, with further performance gains achievable through training. Our code and pre-trained weights are available at https://github.com/Akaneqwq/DualDiff3D.

Related papers

• View-Adaptive Renderer for View-Consistent 2D-to-3D Generation [2.8951915538170288]
  Reconstructing 3D shapes from a single image remains a fundamental yet challenging problem in computer vision.<n>Traditional monocular 3D generation pipelines typically synthesize multiple views from a single input image before applying Neural Radiance Field (NeRF)-based reconstruction.<n>We propose a novel viewpoint-adaptive rendering neural framework that enables robust 3D reconstruction even when given partially inconsistent multi-view inputs.
  arXiv  Detail & Related papers  (2026-08-10T04:28:08Z)
• Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation [58.37682525044409]
  Rewis3d is a framework that leverages recent advances in feed-forward 3D reconstruction to significantly improve weakly supervised semantic segmentation on 2D images.<n>We show that Rewis3d achieves state-of-the-art performance in sparse supervision, outperforming existing approaches by 2-7% without requiring additional labels or inference overhead.
  arXiv  Detail & Related papers  (2026-03-06T15:25:20Z)
• BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model [3.7515646463759698]
  We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos.<n>BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone.<n>We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-02-26T03:58:42Z)
• One-Shot Refiner: Boosting Feed-forward Novel View Synthesis via One-Step Diffusion [57.824020826432815]
  We present a novel framework for high-fidelity novel view synthesis (NVS) from sparse images.<n>We design a Dual-Domain Detail Perception Module, which enables handling high-resolution images without being limited by the ViT backbone.<n>We develop a feature-guided diffusion network, which can preserve high-frequency details during the restoration process.
  arXiv  Detail & Related papers  (2026-01-20T17:11:55Z)
• Splatent: Splatting Diffusion Latents for Novel View Synthesis [6.338499852288994]
  We present Splatent, a diffusion-based enhancement framework designed to operate on top of 3D Gaussian Splatting (3DGS) in the latent space of VAEs.<n>Our key insight departs from the conventional 3D-centric view: rather than reconstructing fine-grained details in 3D space, we recover them in 2D from input views through multi-view attention mechanisms.
  arXiv  Detail & Related papers  (2025-12-10T18:57:04Z)
• Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models [65.90387371072413]
  We introduce Difix3D+, a novel pipeline designed to enhance 3D reconstruction and novel-view synthesis.<n>At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views.
  arXiv  Detail & Related papers  (2025-03-03T17:58:33Z)
• Synthesizing Consistent Novel Views via 3D Epipolar Attention without Re-Training [102.82553402539139]
  Large diffusion models demonstrate remarkable zero-shot capabilities in novel view synthesis from a single image.<n>These models often face challenges in maintaining consistency across novel and reference views.<n>We propose to use epipolar geometry to locate and retrieve overlapping information from the input view.<n>This information is then incorporated into the generation of target views, eliminating the need for training or fine-tuning.
  arXiv  Detail & Related papers  (2025-02-25T14:04:22Z)
• Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views [47.215089338101066]
  We present Sparse3D, a novel 3D reconstruction method tailored for sparse view inputs. Our approach distills robust priors from a multiview-consistent diffusion model to refine a neural radiance field. By tapping into 2D priors from powerful image diffusion models, our integrated model consistently delivers high-quality results.
  arXiv  Detail & Related papers  (2023-08-27T11:52:00Z)
• Generative Novel View Synthesis with 3D-Aware Diffusion Models [96.78397108732233]
  We present a diffusion-based model for 3D-aware generative novel view synthesis from as few as a single input image. Our method makes use of existing 2D diffusion backbones but, crucially, incorporates geometry priors in the form of a 3D feature volume. In addition to generating novel views, our method has the ability to autoregressively synthesize 3D-consistent sequences.
  arXiv  Detail & Related papers  (2023-04-05T17:15:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.