Enhanced Semantic Extraction and Guidance for UGC Image Super Resolution

• URL: http://arxiv.org/abs/2504.09887v1
• Date: Mon, 14 Apr 2025 05:26:24 GMT
• Title: Enhanced Semantic Extraction and Guidance for UGC Image Super Resolution
• Authors: Yiwen Wang, Ying Liang, Yuxuan Zhang, Xinning Chai, Zhengxue Cheng, Yingsheng Qin, Yucai Yang, Rong Xie, Li Song,
• Abstract summary: We propose a novel approach to image super-resolution by integrating semantic guidance into a diffusion framework.<n>Our method addresses the inconsistency between degradations in wild and synthetic datasets.<n>Our model won second place in the CVIRE 2025 Short-form Image Super-Resolution Challenge.
• Score: 18.058473238611725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the disparity between real-world degradations in user-generated content(UGC) images and synthetic degradations, traditional super-resolution methods struggle to generalize effectively, necessitating a more robust approach to model real-world distortions. In this paper, we propose a novel approach to UGC image super-resolution by integrating semantic guidance into a diffusion framework. Our method addresses the inconsistency between degradations in wild and synthetic datasets by separately simulating the degradation processes on the LSDIR dataset and combining them with the official paired training set. Furthermore, we enhance degradation removal and detail generation by incorporating a pretrained semantic extraction model (SAM2) and fine-tuning key hyperparameters for improved perceptual fidelity. Extensive experiments demonstrate the superiority of our approach against state-of-the-art methods. Additionally, the proposed model won second place in the CVPR NTIRE 2025 Short-form UGC Image Super-Resolution Challenge, further validating its effectiveness. The code is available at https://github.c10pom/Moonsofang/NTIRE-2025-SRlab.

Related papers

• Super-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning [20.697932997351813]
  gradient-guided multi-frame super-resolution (MFSR) framework for remote sensing imagery reconstruction. We propose a novel gradient-guided multi-frame super-resolution (MFSR) framework for remote sensing imagery reconstruction.
  arXiv  Detail & Related papers  (2024-12-13T04:19:48Z)
• Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
  Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
  arXiv  Detail & Related papers  (2024-09-25T16:15:21Z)
• Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution [33.16889233975723]
  Implicit degradation modeling-based blind super-resolution (SR) has attracted more increasing attention in the community.<n>We propose a new Content-decoupled Contrastive Learning-based blind image super-resolution (CdCL) framework.
  arXiv  Detail & Related papers  (2024-08-10T04:51:43Z)
• Pairwise Distance Distillation for Unsupervised Real-World Image Super-Resolution [38.79439380482431]
  Real-world super-resolution (RWSR) faces unknown degradations in the low-resolution inputs, all the while lacking paired training data. Existing methods approach this problem by learning blind general models through complex synthetic augmentations on training inputs. We introduce a novel pairwise distance distillation framework to address the unsupervised RWSR for a targeted real-world degradation.
  arXiv  Detail & Related papers  (2024-07-10T01:46:40Z)
• IG-CFAT: An Improved GAN-Based Framework for Effectively Exploiting Transformers in Real-World Image Super-Resolution [2.1561701531034414]
  Recently, composite fusion attention transformer (CFAT) outperformed previous state-of-the-art (SOTA) models in classic image super-resolution.<n>In this paper, we propose a novel GAN-based framework by incorporating the CFAT model to effectively exploit the performance of transformers in real-world image super-resolution.
  arXiv  Detail & Related papers  (2024-06-19T20:21:26Z)
• DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion [27.52552274944687]
  We introduce a novel two-stage, degradation-aware framework that enhances the diffusion model's ability to recognize content and degradation in low-resolution images. In the first stage, we employ unsupervised contrastive learning to obtain representations of image degradations. In the second stage, we integrate a degradation-aware module into a simplified ControlNet, enabling flexible adaptation to various degradations.
  arXiv  Detail & Related papers  (2024-03-31T12:07:04Z)
• CasSR: Activating Image Power for Real-World Image Super-Resolution [24.152495730507823]
  Cascaded diffusion for Super-Resolution, CasSR, is a novel method designed to produce highly detailed and realistic images. We develop a cascaded controllable diffusion model that aims to optimize the extraction of information from low-resolution images.
  arXiv  Detail & Related papers  (2024-03-18T03:59:43Z)
• LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement [118.83316133601319]
  Current deep learning methods for low-light image enhancement (LLIE) typically rely on pixel-wise mapping learned from paired data. We propose a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process.
  arXiv  Detail & Related papers  (2023-07-27T07:22:51Z)
• ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution [84.73658185158222]
  We propose a diffusion model-based super-resolution method called ACDMSR. Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
  arXiv  Detail & Related papers  (2023-07-03T06:49:04Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• Exploiting Diffusion Prior for Real-World Image Super-Resolution [75.5898357277047]
  We present a novel approach to leverage prior knowledge encapsulated in pre-trained text-to-image diffusion models for blind super-resolution. By employing our time-aware encoder, we can achieve promising restoration results without altering the pre-trained synthesis model.
  arXiv  Detail & Related papers  (2023-05-11T17:55:25Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.