Advancing Super-Resolution in Neural Radiance Fields via Variational Diffusion Strategies

• URL: http://arxiv.org/abs/2410.18137v1
• Date: Tue, 22 Oct 2024 00:02:26 GMT
• Title: Advancing Super-Resolution in Neural Radiance Fields via Variational Diffusion Strategies
• Authors: Shrey Vishen, Jatin Sarabu, Chinmay Bharathulwar, Rithwick Lakshmanan, Vishnu Srinivas,
• Abstract summary: We present a novel method for diffusion-guided frameworks for view-consistent super-resolution (SR) in neural rendering. Our approach leverages existing 2D SR models in conjunction with advanced techniques such as Variational Score Distilling (VSD) and a LoRA fine-tuning helper.
• Score: 0.0
• License:
• Abstract: We present a novel method for diffusion-guided frameworks for view-consistent super-resolution (SR) in neural rendering. Our approach leverages existing 2D SR models in conjunction with advanced techniques such as Variational Score Distilling (VSD) and a LoRA fine-tuning helper, with spatial training to significantly boost the quality and consistency of upscaled 2D images compared to the previous methods in the literature, such as Renoised Score Distillation (RSD) proposed in DiSR-NeRF (1), or SDS proposed in DreamFusion. The VSD score facilitates precise fine-tuning of SR models, resulting in high-quality, view-consistent images. To address the common challenge of inconsistencies among independent SR 2D images, we integrate Iterative 3D Synchronization (I3DS) from the DiSR-NeRF framework. Our quantitative benchmarks and qualitative results on the LLFF dataset demonstrate the superior performance of our system compared to existing methods such as DiSR-NeRF.

Related papers

• Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
  Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
  arXiv  Detail & Related papers  (2024-09-25T16:15:21Z)
• ASSR-NeRF: Arbitrary-Scale Super-Resolution on Voxel Grid for High-Quality Radiance Fields Reconstruction [27.21399221644529]
  NeRF-based methods reconstruct 3D scenes by building a radiance field with implicit or explicit representations. We propose Arbitrary-Scale Super-Resolution NeRF (ASSR-NeRF), a novel framework for super-resolution novel view synthesis (SRNVS)
  arXiv  Detail & Related papers  (2024-06-28T17:22:33Z)
• DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF [50.458896463542494]
  DiSR-NeRF is a diffusion-guided framework for view-consistent super-resolution (SR) NeRF. We propose Iterative 3D Synchronization (I3DS) to mitigate the inconsistency problem via the inherent multi-view consistency property of NeRF.
  arXiv  Detail & Related papers  (2024-04-01T03:06:23Z)
• DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution [88.13972071356422]
  We propose a diffusion-style data augmentation scheme for GAN-based image super-resolution (SR) methods, known as DifAugGAN. It involves adapting the diffusion process in generative diffusion models for improving the calibration of the discriminator during training. Our DifAugGAN can be a Plug-and-Play strategy for current GAN-based SISR methods to improve the calibration of the discriminator and thus improve SR performance.
  arXiv  Detail & Related papers  (2023-11-30T12:37:53Z)
• S2R: Exploring a Double-Win Transformer-Based Framework for Ideal and Blind Super-Resolution [5.617008573997855]
  A light-weight transformer-based SR model (S2R transformer) and a novel coarse-to-fine training strategy are proposed. The proposed S2R outperforms other single-image SR models in ideal SR condition with only 578K parameters. It can achieve better visual results than regular blind SR models in blind fuzzy conditions with only 10 gradient updates.
  arXiv  Detail & Related papers  (2023-08-16T04:27:44Z)
• RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution [57.98314517861539]
  Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images. In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network.
  arXiv  Detail & Related papers  (2023-06-30T12:14:13Z)
• Perception-Distortion Trade-off in the SR Space Spanned by Flow Models [21.597478894658263]
  Flow-based generative super-resolution (SR) models learn to produce a diverse set of feasible SR solutions, called the SR space. We present a simple but effective image ensembling/fusion approach to obtain a single SR image eliminating random artifacts and improving fidelity without significantly compromising perceptual quality.
  arXiv  Detail & Related papers  (2022-09-18T13:12:21Z)
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models [19.17571465274627]
  Single image super-resolution (SISR) aims to reconstruct high-resolution (HR) images from the given low-resolution (LR) ones. We propose a novel single image super-resolution diffusion probabilistic model (SRDiff) SRDiff is optimized with a variant of the variational bound on the data likelihood and can provide diverse and realistic SR predictions.
  arXiv  Detail & Related papers  (2021-04-30T12:31:25Z)
• Accurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network [63.69237156340457]
  We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN) MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations) The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
  arXiv  Detail & Related papers  (2020-09-14T08:23:37Z)
• Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
  Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data. This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. We formulate a method that joins the advantages of conventional SR and UDA models.
  arXiv  Detail & Related papers  (2019-12-30T16:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.