DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning

• URL: http://arxiv.org/abs/2505.17910v1
• Date: Fri, 23 May 2025 13:53:23 GMT
• Title: DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning
• Authors: Bin Wu, Wei Wang, Yahui Liu, Zixiang Li, Yao Zhao,
• Abstract summary: We introduce a ReFL framework, named DiffusionReward, to the Blind Face Restoration task for the first time.<n>The core of our framework is the Face Reward Model (FRM), which is trained using carefully annotated data.<n>Experiments on synthetic and wild datasets demonstrate that our method outperforms state-of-the-art methods.
• Score: 40.641049729447175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reward Feedback Learning (ReFL) has recently shown great potential in aligning model outputs with human preferences across various generative tasks. In this work, we introduce a ReFL framework, named DiffusionReward, to the Blind Face Restoration task for the first time. DiffusionReward effectively overcomes the limitations of diffusion-based methods, which often fail to generate realistic facial details and exhibit poor identity consistency. The core of our framework is the Face Reward Model (FRM), which is trained using carefully annotated data. It provides feedback signals that play a pivotal role in steering the optimization process of the restoration network. In particular, our ReFL framework incorporates a gradient flow into the denoising process of off-the-shelf face restoration methods to guide the update of model parameters. The guiding gradient is collaboratively determined by three aspects: (i) the FRM to ensure the perceptual quality of the restored faces; (ii) a regularization term that functions as a safeguard to preserve generative diversity; and (iii) a structural consistency constraint to maintain facial fidelity. Furthermore, the FRM undergoes dynamic optimization throughout the process. It not only ensures that the restoration network stays precisely aligned with the real face manifold, but also effectively prevents reward hacking. Experiments on synthetic and wild datasets demonstrate that our method outperforms state-of-the-art methods, significantly improving identity consistency and facial details. The source codes, data, and models are available at: https://github.com/01NeuralNinja/DiffusionReward.

Related papers

• LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter [52.93785843453579]
  Blind face restoration from low-quality (LQ) images is a challenging task that requires high-fidelity image reconstruction and the preservation of facial identity.<n>We propose LAFR, a novel codebook-based latent space adapter that aligns the latent distribution of LQ images with that of HQ counterparts.<n>We show that lightweight finetuning of diffusion prior on just 0.9% of FFHQ dataset is sufficient to achieve results comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-29T14:11:16Z)
• Reference-Guided Identity Preserving Face Restoration [54.10295747851343]
  Preserving face identity is a critical yet persistent challenge in diffusion-based image restoration.<n>This paper introduces a novel approach that maximizes reference face utility for improved face restoration and identity preservation.
  arXiv  Detail & Related papers  (2025-05-28T02:46:34Z)
• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration.<n>We propose OSDFace, a novel one-step diffusion model for face restoration.<n>Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• 3D Priors-Guided Diffusion for Blind Face Restoration [30.94188504133298]
  Blind face restoration endeavors to restore a clear face image from a degraded counterpart. Recent approaches employing Generative Adversarial Networks (GANs) as priors have demonstrated remarkable success. We propose a novel diffusion-based framework by embedding the 3D facial priors as structure and identity constraints into a denoising diffusion process.
  arXiv  Detail & Related papers  (2024-09-02T07:13:32Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• BFRFormer: Transformer-based generator for Real-World Blind Face Restoration [37.77996097891398]
  We propose a Transformer-based blind face restoration method, named BFRFormer, to reconstruct images with more identity-preserved details in an end-to-end manner. Our method outperforms state-of-the-art methods on a synthetic dataset and four real-world datasets.
  arXiv  Detail & Related papers  (2024-02-29T02:31:54Z)
• CLR-Face: Conditional Latent Refinement for Blind Face Restoration Using Score-Based Diffusion Models [57.9771859175664]
  Recent generative-prior-based methods have shown promising blind face restoration performance. Generating fine-grained facial details faithful to inputs remains a challenging problem. We introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings.
  arXiv  Detail & Related papers  (2024-02-08T23:51:49Z)
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• DifFace: Blind Face Restoration with Diffused Error Contraction [62.476329680424975]
  DifFace is capable of coping with unseen and complex degradations more gracefully without complicated loss designs. It is superior to current state-of-the-art methods, especially in cases with severe degradations.
  arXiv  Detail & Related papers  (2022-12-13T11:52:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.