Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Models

• URL: http://arxiv.org/abs/2306.00714v1
• Date: Thu, 1 Jun 2023 14:20:06 GMT
• Title: Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Models
• Abstract summary: Diffusion-based Generative Models (DGMs) have achieved unparalleled performance in synthesizing high-quality visual content. Recent solutions for these tasks often train architecture-specific DGMs from scratch, or require iterative fine-tuning and distillation on pre-trained DGMs. We propose Diff-SR, a first ASSR attempt based solely on pre-trained DGMs, without additional training efforts.
• Score: 24.82036158180386
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based Generative Models (DGMs) have achieved unparalleled performance in synthesizing high-quality visual content, opening up the opportunity to improve image super-resolution (SR) tasks. Recent solutions for these tasks often train architecture-specific DGMs from scratch, or require iterative fine-tuning and distillation on pre-trained DGMs, both of which take considerable time and hardware investments. More seriously, since the DGMs are established with a discrete pre-defined upsampling scale, they cannot well match the emerging requirements of arbitrary-scale super-resolution (ASSR), where a unified model adapts to arbitrary upsampling scales, instead of preparing a series of distinct models for each case. These limitations beg an intriguing question: can we identify the ASSR capability of existing pre-trained DGMs without the need for distillation or fine-tuning? In this paper, we take a step towards resolving this matter by proposing Diff-SR, a first ASSR attempt based solely on pre-trained DGMs, without additional training efforts. It is motivated by an exciting finding that a simple methodology, which first injects a specific amount of noise into the low-resolution images before invoking a DGM's backward diffusion process, outperforms current leading solutions. The key insight is determining a suitable amount of noise to inject, i.e., small amounts lead to poor low-level fidelity, while over-large amounts degrade the high-level signature. Through a finely-grained theoretical analysis, we propose the Perceptual Recoverable Field (PRF), a metric that achieves the optimal trade-off between these two factors. Extensive experiments verify the effectiveness, flexibility, and adaptability of Diff-SR, demonstrating superior performance to state-of-the-art solutions under diverse ASSR environments.

Related papers

• Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching [36.00855152229835]
  We propose a unified, single-stage optimization framework grounded in Distribution Matching.<n>In the standard DM framework, distillation updates the model via a gradient direction that minimizes the gap between the real and fake models.<n>We introduce DM-Align, which derives a complementary gradient direction to guide the model toward human-preferred samples.
  arXiv  Detail & Related papers  (2026-09-03T04:46:46Z)
• Efficient Difficulty-Aware Dynamic Routing for Diffusion-Based Real-World Image Super-Resolution [69.17533212524447]
  We propose a Difficulty-aware Dynamic Routing ( DDR) strategy that overcomes the rigid, one-size-fits-all processing paradigm.<n>Specifically, we design a difficulty estimator to predict the restoration cost of each input image, enabling automatic assignment to a network of appropriate capacity.<n>Extensive experiments have demonstrated the superior efficiency and effectiveness of the proposed model compared to recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-17T07:49:05Z)
• DTI: Dynamic Trajectory Initialization for Generative Face Video Super-Resolution [12.049894329245404]
  We present Dynamic Trajectory Initialization (DTI) paradigm for Face Video Super-Resolution (FVSR)<n>With a novel enhancement-and-injection conditioning mechanism, fidelity of our model has been significantly improved without compromising perceptual quality.<n>Our method achieves a SOTA overall performance across diverse metrics and benchmarks.
  arXiv  Detail & Related papers  (2026-06-28T04:55:10Z)
• Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection [52.5174167737992]
  Video anomaly detection (VAD) aims to identify abnormal events in videos.<n>We propose SteerVAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations.<n>Our method achieves state-of-the-art performance among tuning-free approaches requiring only 1% of training data.
  arXiv  Detail & Related papers  (2026-02-27T13:48:50Z)
• From Structure to Detail: Hierarchical Distillation for Efficient Diffusion Model [18.782919607372328]
  Trajectory-based and distribution-based step distillation methods offer solutions.<n>Trajectory-based methods preserve global structure but act as a "lossy compressor"<n>We recast them into synergistic components within our novel Hierarchical Distillation framework.
  arXiv  Detail & Related papers  (2025-11-12T03:12:06Z)
• RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance [101.30279597148973]
  We propose reward-free guidance (RFG) for guiding the reasoning trajectory of dLLMs without explicit process reward.<n>RFG consistently yields significant improvements across all tasks and model types, achieving accuracy gains of up to 9.2%.
  arXiv  Detail & Related papers  (2025-09-29T23:59:16Z)
• Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
  arXiv  Detail & Related papers  (2025-05-19T17:21:03Z)
• Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution [10.074968164380314]
  Implicit Neural Representation (INR) has been successfully employed for Arbitrary-scale Super-Resolution (ASR) We develop two novel techniques to generalize GS for ASR. We implement an efficient differentiable 2D GPU/CUDA-based scale-awareization to render super-aware images.
  arXiv  Detail & Related papers  (2025-01-12T15:14:58Z)
• Latent Diffusion, Implicit Amplification: Efficient Continuous-Scale Super-Resolution for Remote Sensing Images [7.920423405957888]
  E$2$DiffSR achieves superior objective metrics and visual quality compared to the state-of-the-art SR methods. It reduces the inference time of diffusion-based SR methods to a level comparable to that of non-diffusion methods.
  arXiv  Detail & Related papers  (2024-10-30T09:14:13Z)
• Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
  Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
  arXiv  Detail & Related papers  (2024-09-25T16:15:21Z)
• Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution [18.71638301931374]
  generative priors of pre-trained latent diffusion models (DMs) have demonstrated great potential to enhance the visual quality of image super-resolution (SR) results. We propose to partition the generative SR process into two stages, where the DM is employed for reconstructing image structures and the GAN is employed for improving fine-grained details. Once trained, our proposed method, namely content consistent super-resolution (CCSR),allows flexible use of different diffusion steps in the inference stage without re-training.
  arXiv  Detail & Related papers  (2023-12-30T10:22:59Z)
• DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution [88.13972071356422]
  We propose a diffusion-style data augmentation scheme for GAN-based image super-resolution (SR) methods, known as DifAugGAN. It involves adapting the diffusion process in generative diffusion models for improving the calibration of the discriminator during training. Our DifAugGAN can be a Plug-and-Play strategy for current GAN-based SISR methods to improve the calibration of the discriminator and thus improve SR performance.
  arXiv  Detail & Related papers  (2023-11-30T12:37:53Z)
• Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization [61.39201891894024]
  Group distributionally robust optimization (group DRO) can minimize the worst-case loss over pre-defined groups. We reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization.
  arXiv  Detail & Related papers  (2023-05-20T07:02:27Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Implicit Diffusion Models for Continuous Super-Resolution [65.45848137914592]
  This paper introduces an Implicit Diffusion Model (IDM) for high-fidelity continuous image super-resolution. IDM integrates an implicit neural representation and a denoising diffusion model in a unified end-to-end framework. The scaling factor regulates the resolution and accordingly modulates the proportion of the LR information and generated features in the final output.
  arXiv  Detail & Related papers  (2023-03-29T07:02:20Z)
• Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling [36.02321871608158]
  We propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the problem. PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to 29x on more challenging high resolution (1024x1024) image generation.
  arXiv  Detail & Related papers  (2022-07-05T17:55:42Z)
• Accelerating Score-based Generative Models for High-Resolution Image Synthesis [42.076244561541706]
  Score-based generative models (SGMs) have recently emerged as a promising class of generative models. In this work, we consider the acceleration of high-resolution generation with SGMs. We introduce a novel Target Distribution Sampling Aware (TDAS) method by leveraging the structural priors in space and frequency domains.
  arXiv  Detail & Related papers  (2022-06-08T17:41:14Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.