Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration

• URL: http://arxiv.org/abs/2603.16570v1
• Date: Tue, 17 Mar 2026 14:27:24 GMT
• Title: Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration
• Abstract summary: Face2Scene is a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image.<n>Experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.
• Score: 22.308112548367728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in image restoration have enabled high-fidelity recovery of faces from degraded inputs using reference-based face restoration models (Ref-FR). However, such methods focus solely on facial regions, neglecting degradation across the full scene, including body and background, which limits practical usability. Meanwhile, full-scene restorers often ignore degradation cues entirely, leading to underdetermined predictions and visual artifacts. In this work, we propose Face2Scene, a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image. Given a degraded image and one or more identity references, we first apply a Ref-FR model to reconstruct high-quality facial details. From the restored-degraded face pair, we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression), which is then transformed into multi-scale degradation-aware tokens. These tokens condition a diffusion model to restore the full scene in a single step, including the body and background. Extensive experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.

Related papers

• Visual-Instructed Degradation Diffusion for All-in-One Image Restoration [29.910376294021052]
  We propose textbfDefusion, a novel all-in-one image restoration framework that utilizes visual instruction-guided degradation diffusion.<n>We show that Defusion outperforms state-of-the-art methods across diverse image restoration tasks, including complex and real-world degradations.
  arXiv  Detail & Related papers  (2025-06-20T12:50:42Z)
• Invert2Restore: Zero-Shot Degradation-Blind Image Restoration [19.263005158979567]
  Invert2Restore is a zero-shot, training-free method that operates in both fully blind and partially blind settings.<n>It generalizes well across various types of image degradation.<n>We experimentally validate Invert2Restore across several image restoration tasks.
  arXiv  Detail & Related papers  (2025-03-27T13:22:40Z)
• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration.<n>We propose OSDFace, a novel one-step diffusion model for face restoration.<n>Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior [13.27748226506837]
  Blind face restoration (BFR) is a fundamental and challenging problem in computer vision. Recent research endeavors rely on facial image priors from the powerful pretrained text-to-image (T2I) diffusion models. We propose AuthFace, which achieves highly authentic face restoration results by exploring a face-oriented generative diffusion prior.
  arXiv  Detail & Related papers  (2024-10-13T14:56:13Z)
• Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model [55.46927355649013]
  We introduce a novel Multi-modal Guided Real-World Face Restoration technique.<n>MGFR can mitigate the generation of false facial attributes and identities.<n>We present the Reface-HQ dataset, comprising over 21,000 high-resolution facial images across 4800 identities.
  arXiv  Detail & Related papers  (2024-10-05T13:46:56Z)
• Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration [19.87693298262894]
  We propose Diff-Restorer, a universal image restoration method based on the diffusion model. We utilize the pre-trained visual language model to extract visual prompts from degraded images. We also design a Degradation-aware Decoder to perform structural correction and convert the latent code to the pixel domain.
  arXiv  Detail & Related papers  (2024-07-04T05:01:10Z)
• RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs [63.991802204929485]
  Blind face restoration aims at recovering high-quality face images from those with unknown degradations. Current algorithms mainly introduce priors to complement high-quality details and achieve impressive progress. We propose RestoreFormer++, which introduces fully-spatial attention mechanisms to model the contextual information and the interplay with the priors. We show that RestoreFormer++ outperforms state-of-the-art algorithms on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-08-14T16:04:53Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Implicit Subspace Prior Learning for Dual-Blind Face Restoration [66.67059961379923]
  A novel implicit subspace prior learning (ISPL) framework is proposed as a generic solution to dual-blind face restoration. Experimental results demonstrate significant perception-distortion improvement of ISPL against existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-12T08:04:24Z)
• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [63.333407973913374]
  "Face Renovation"(FR) is a semantic-guided generation problem. "HiFaceGAN" is a multi-stage framework containing several nested CSR units. experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
  arXiv  Detail & Related papers  (2020-05-11T11:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.