InfoBFR: Real-World Blind Face Restoration via Information Bottleneck

• URL: http://arxiv.org/abs/2501.15443v1
• Date: Sun, 26 Jan 2025 08:11:52 GMT
• Title: InfoBFR: Real-World Blind Face Restoration via Information Bottleneck
• Authors: Nan Gao, Jia Li, Huaibo Huang, Ke Shang, Ran He,
• Abstract summary: We propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.
• Score: 66.14954670338967
• License:
• Abstract: Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of data degradation patterns. Current BFR methods have realized certain restored productions but with inherent neural degradations that limit real-world generalization in complicated scenarios. In this paper, we propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues, which achieves high-generalization face restoration in diverse wild and heterogeneous scenes. Specifically, based on the results from pre-trained BFR models, InfoBFR considers information compression using manifold information bottleneck (MIB) and information compensation with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. Comprehensive experimental results demonstrate the superiority of InfoBFR over state-of-the-art GAN-based and diffusion-based BFR methods, with around 70ms consumption, 16M trainable parameters, and nearly 85% BFR-boosting. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.

Related papers

• DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration [7.521850476177286]
  We equip diffusion models with the capability to decouple various degradation as a degradation prompt from low-quality (LQ) face images. Our novel restoration scheme, named DR-BFR, guides the denoising of Latent Diffusion Models (LDM) by incorporating Degradation Representation (DR) and content features from LQ images. DR-BFR significantly outperforms state-of-the-art methods quantitatively and qualitatively across various datasets.
  arXiv  Detail & Related papers  (2024-11-15T15:24:42Z)
• IB-AdCSCNet:Adaptive Convolutional Sparse Coding Network Driven by Information Bottleneck [4.523653503622693]
  We introduce IB-AdCSCNet, a deep learning model grounded in information bottleneck theory. IB-AdCSCNet seamlessly integrates the information bottleneck trade-off strategy into deep networks. Experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate that IB-AdCSCNet not only matches the performance of deep residual convolutional networks but also outperforms them when handling corrupted data.
  arXiv  Detail & Related papers  (2024-05-23T05:35:57Z)
• Towards Real-world Video Face Restoration: A New Benchmark [33.01372704755186]
  We introduce new real-world datasets named FOS with a taxonomy of "Full, Occluded, and Side" faces. FOS datasets cover more diverse degradations and involve face samples from more complex scenarios. We benchmarked both the state-of-the-art BFR methods and the video super resolution (VSR) methods to comprehensively study current approaches.
  arXiv  Detail & Related papers  (2024-04-30T12:37:01Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration [62.44659039265439]
  We propose a Diffusion-Information-Diffusion framework to tackle blind face restoration. DiffMAC achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Results demonstrate the superiority of DiffMAC over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-15T08:44:15Z)
• BFRFormer: Transformer-based generator for Real-World Blind Face Restoration [37.77996097891398]
  We propose a Transformer-based blind face restoration method, named BFRFormer, to reconstruct images with more identity-preserved details in an end-to-end manner. Our method outperforms state-of-the-art methods on a synthetic dataset and four real-world datasets.
  arXiv  Detail & Related papers  (2024-02-29T02:31:54Z)
• RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution [57.98314517861539]
  Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images. In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network.
  arXiv  Detail & Related papers  (2023-06-30T12:14:13Z)
• Generative Diffusion Prior for Unified Image Restoration and Enhancement [62.76390152617949]
  Existing image restoration methods mostly leverage the posterior distribution of natural images. We propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems.
  arXiv  Detail & Related papers  (2023-04-03T16:52:43Z)
• Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration [4.244692655670362]
  Panini-Net is a degradation-aware feature network for face restoration. It learns the abstract representations to distinguish various degradations. It achieves state-of-the-art performance for multi-degradation face restoration and face super-resolution.
  arXiv  Detail & Related papers  (2022-03-16T07:41:07Z)
• A Generalizable Model-and-Data Driven Approach for Open-Set RFF Authentication [74.63333951647581]
  Radio-frequency fingerprints(RFFs) are promising solutions for realizing low-cost physical layer authentication. Machine learning-based methods have been proposed for RFF extraction and discrimination. We propose a new end-to-end deep learning framework for extracting RFFs from raw received signals.
  arXiv  Detail & Related papers  (2021-08-10T03:59:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.