DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction

• URL: http://arxiv.org/abs/2509.17232v1
• Date: Sun, 21 Sep 2025 20:45:22 GMT
• Title: DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction
• Abstract summary: This paper proposes a Diffusion Model-d Neural Radiance Field (DT-NeRF) method to enhance detail recovery and multi-view consistency in 3D scene reconstruction.<n>By combining diffusion models with Transformers, DT-NeRF effectively restores details under sparse viewpoints and maintains high accuracy in complex geometric scenes.
• Score: 13.622738176769188
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a Diffusion Model-Optimized Neural Radiance Field (DT-NeRF) method, aimed at enhancing detail recovery and multi-view consistency in 3D scene reconstruction. By combining diffusion models with Transformers, DT-NeRF effectively restores details under sparse viewpoints and maintains high accuracy in complex geometric scenes. Experimental results demonstrate that DT-NeRF significantly outperforms traditional NeRF and other state-of-the-art methods on the Matterport3D and ShapeNet datasets, particularly in metrics such as PSNR, SSIM, Chamfer Distance, and Fidelity. Ablation experiments further confirm the critical role of the diffusion and Transformer modules in the model's performance, with the removal of either module leading to a decline in performance. The design of DT-NeRF showcases the synergistic effect between modules, providing an efficient and accurate solution for 3D scene reconstruction. Future research may focus on further optimizing the model, exploring more advanced generative models and network architectures to enhance its performance in large-scale dynamic scenes.

Related papers

• Wavelet-Optimized Pseudo-3D Accelerated Diffusion Model for Truncated Computed Laminography [12.168086376276934]
  Field-of-view (FOV) limitations inevitably lead to truncation of projected data.<n>Existing deep learning methods rely on 2D architectures that lack rigorous data consistency constraints.<n>We propose a novel wavelet-optimized pseudo-3D accelerated diffusion model for CL truncation reconstruction (CL-DM)
  arXiv  Detail & Related papers  (2026-06-30T08:24:57Z)
• Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation [20.986333949052007]
  Pixel-Level Residual Diffusion Transformer (PRDiT) is an inherent generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level.<n>PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.
  arXiv  Detail & Related papers  (2026-06-18T11:35:11Z)
• DiffNR: Diffusion-Enhanced Neural Representation Optimization for Sparse-View 3D Tomographic Reconstruction [42.4365911571903]
  We propose DiffNR, a novel framework that enhances NR optimization with diffusion priors.<n>At its core is SliceFixer, a single-step diffusion model designed to correct artifacts in degraded slices.<n>In experiments, DiffNR improves PSNR by 3.99 dB on average, generalizes well across domains, and maintains efficient optimization.
  arXiv  Detail & Related papers  (2026-04-23T10:27:54Z)
• A Neural Network Architecture Based on Attention Gate Mechanism for 3D Magnetotelluric Forward Modeling [1.5862483908050367]
  We propose a novel neural network architecture named MTAGU-Net, which integrates an attention gating mechanism for 3D MT forward modeling.<n>A dual-path attention gating module is designed based on forward response data images and embedded in the skip connections between the encoder and decoder.<n>A synthetic model generation method utilizing 3D Gaussian random field (GRF) accurately replicates the electrical structures of real-world geological scenarios.
  arXiv  Detail & Related papers  (2025-03-14T13:48:25Z)
• DRDT3: Diffusion-Refined Decision Test-Time Training Model [9.181158786602085]
  Decision Transformer (DT) has shown competitive performance compared to traditional offline reinforcement learning (RL) approaches.<n>We introduce a unified framework, called Diffusion-Refined Decision TTT (DRDT3), to achieve performance beyond DT models.
  arXiv  Detail & Related papers  (2025-01-12T04:59:49Z)
• A Staged Deep Learning Approach to Spatial Refinement in 3D Temporal Atmospheric Transport [0.0]
  We introduce the Dual-Stage Temporal Three-dimensional Super-resolution (DST3D-UNet-SR) model for plume dispersion prediction.<n>It is composed of two sequential modules: the temporal module (TM), which predicts the transient evolution of a plume in complex terrain from low-resolution temporal data, and the spatial refinement module (SRM), which subsequently enhances the spatial resolution of the predictions.
  arXiv  Detail & Related papers  (2024-12-14T19:43:48Z)
• 4Diffusion: Multi-view Video Diffusion Model for 4D Generation [55.82208863521353]
  Current 4D generation methods have achieved noteworthy efficacy with the aid of advanced diffusion generative models. We propose a novel 4D generation pipeline, namely 4Diffusion, aimed at generating spatial-temporally consistent 4D content from a monocular video.
  arXiv  Detail & Related papers  (2024-05-31T08:18:39Z)
• Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation [51.346733271166926]
  Mesh2NeRF is an approach to derive ground-truth radiance fields from textured meshes for 3D generation tasks. We validate the effectiveness of Mesh2NeRF across various tasks.
  arXiv  Detail & Related papers  (2024-03-28T11:22:53Z)
• SeisFusion: Constrained Diffusion Model with Input Guidance for 3D Seismic Data Interpolation and Reconstruction [26.02191880837226]
  We propose a novel diffusion model reconstruction framework tailored for 3D seismic data. We introduce a 3D neural network architecture into the diffusion model, successfully extending the 2D diffusion model to 3D space. Our method exhibits superior reconstruction accuracy when applied to both field datasets and synthetic datasets.
  arXiv  Detail & Related papers  (2024-03-18T05:10:13Z)
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D [88.66678730537777]
  We present StableDreamer, a methodology incorporating three advances. First, we formalize the equivalence of the SDS generative prior and a simple supervised L2 reconstruction loss. Second, our analysis shows that while image-space diffusion contributes to geometric precision, latent-space diffusion is crucial for vivid color rendition.
  arXiv  Detail & Related papers  (2023-12-02T02:27:58Z)
• Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models [8.590026259176806]
  We propose a diffusion-model-based framework that overcomes the limitations of requiring reference data or prior knowledge about the degradation process. Our approach utilizes 2D diffusion models to consistently reconstruct 3D volumes and is well-suited for highly downsampled data.
  arXiv  Detail & Related papers  (2023-08-03T07:57:02Z)
• Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction [77.69363640021503]
  3D-aware image synthesis encompasses a variety of tasks, such as scene generation and novel view synthesis from images. We present SSDNeRF, a unified approach that employs an expressive diffusion model to learn a generalizable prior of neural radiance fields (NeRF) from multi-view images of diverse objects.
  arXiv  Detail & Related papers  (2023-04-13T17:59:01Z)
• DehazeNeRF: Multiple Image Haze Removal and 3D Shape Reconstruction using Neural Radiance Fields [56.30120727729177]
  We introduce DehazeNeRF as a framework that robustly operates in hazy conditions. We demonstrate successful multi-view haze removal, novel view synthesis, and 3D shape reconstruction where existing approaches fail.
  arXiv  Detail & Related papers  (2023-03-20T18:03:32Z)
• Accurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network [63.69237156340457]
  We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN) MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations) The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
  arXiv  Detail & Related papers  (2020-09-14T08:23:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.