Multi-Feature Aggregation in Diffusion Models for Enhanced Face Super-Resolution

• URL: http://arxiv.org/abs/2408.15386v2
• Date: Sun, 20 Oct 2024 15:13:54 GMT
• Title: Multi-Feature Aggregation in Diffusion Models for Enhanced Face Super-Resolution
• Authors: Marcelo dos Santos, Rayson Laroca, Rafael O. Ribeiro, João C. Neves, David Menotti,
• Abstract summary: We develop an algorithm that utilize a low-resolution image combined with features extracted from multiple low-quality images to generate a super-resolved image. Unlike other algorithms, our approach recovers facial features without explicitly providing attribute information. This is the first time multi-features combined with low-resolution images are used as conditioners to generate more reliable super-resolution images.
• Score: 6.055006354743854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Super-resolution algorithms often struggle with images from surveillance environments due to adverse conditions such as unknown degradation, variations in pose, irregular illumination, and occlusions. However, acquiring multiple images, even of low quality, is possible with surveillance cameras. In this work, we develop an algorithm based on diffusion models that utilize a low-resolution image combined with features extracted from multiple low-quality images to generate a super-resolved image while minimizing distortions in the individual's identity. Unlike other algorithms, our approach recovers facial features without explicitly providing attribute information or without the need to calculate a gradient of a function during the reconstruction process. To the best of our knowledge, this is the first time multi-features combined with low-resolution images are used as conditioners to generate more reliable super-resolution images using stochastic differential equations. The FFHQ dataset was employed for training, resulting in state-of-the-art performance in facial recognition and verification metrics when evaluated on the CelebA and Quis-Campi datasets. Our code is publicly available at https://github.com/marcelowds/fasr

Related papers

• Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Network [8.739451985459638]
  Super-resolution algorithms transform one or more sets of low-resolution images captured from the same scene into high-resolution images. The extraction of image features and nonlinear mapping methods in the reconstruction process remain challenging for existing algorithms. The objective is to recover high-quality, high-resolution images from low-resolution images.
  arXiv  Detail & Related papers  (2024-07-18T06:50:39Z)
• Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder [29.924160271522354]
  Super-resolution (SR) and image generation are important tasks in computer vision and are widely adopted in real-world applications. Most existing methods, however, generate images only at fixed-scale magnification and suffer from over-smoothing and artifacts. Most relevant work applied Implicit Neural Representation (INR) to the denoising diffusion model to obtain continuous-resolution yet diverse and high-quality SR results. We propose a novel pipeline that can super-resolve an input image or generate from a random noise a novel image at arbitrary scales.
  arXiv  Detail & Related papers  (2024-03-15T12:45:40Z)
• Deep Richardson-Lucy Deconvolution for Low-Light Image Deblurring [48.80983873199214]
  We develop a data-driven approach to model the saturated pixels by a learned latent map. Based on the new model, the non-blind deblurring task can be formulated into a maximum a posterior (MAP) problem. To estimate high-quality deblurred images without amplified artifacts, we develop a prior estimation network.
  arXiv  Detail & Related papers  (2023-08-10T12:53:30Z)
• Exploring Resolution and Degradation Clues as Self-supervised Signal for Low Quality Object Detection [77.3530907443279]
  We propose a novel self-supervised framework to detect objects in degraded low resolution images. Our methods has achieved superior performance compared with existing methods when facing variant degradation situations.
  arXiv  Detail & Related papers  (2022-08-05T09:36:13Z)
• Pyramidal Denoising Diffusion Probabilistic Models [43.9925721757248]
  We present a novel pyramidal diffusion model to generate high resolution images using a single score function trained with a positional embedding. This enables a time-efficient sampling for image generation, and also solves the low batch size problem when training with limited resources.
  arXiv  Detail & Related papers  (2022-08-03T06:26:18Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
  We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms. Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-27T12:12:57Z)
• Image Inpainting with Learnable Feature Imputation [8.293345261434943]
  A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. We propose (layer-wise) feature imputation of the missing input values to a convolution. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model.
  arXiv  Detail & Related papers  (2020-11-02T16:05:32Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.