Latent Posterior-Mean Rectified Flow for Higher-Fidelity Perceptual Face Restoration

• URL: http://arxiv.org/abs/2507.00447v1
• Date: Tue, 01 Jul 2025 06:00:28 GMT
• Title: Latent Posterior-Mean Rectified Flow for Higher-Fidelity Perceptual Face Restoration
• Authors: Xin Luo, Menglin Zhang, Yunwei Lan, Tianyu Zhang, Rui Li, Chang Liu, Dong Liu,
• Abstract summary: Posterior-Mean Rectified Flow (PMRF) proposes a flow based approach where source distribution is minimum distortion estimations.<n>Latent-PMRF reformulates PMRF in the latent space of a variational autoencoder (VAE)<n>Our proposed VAE significantly outperforms existing VAEs in both reconstruction and restoration.
• Score: 17.497971830313883
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Perception-Distortion tradeoff (PD-tradeoff) theory suggests that face restoration algorithms must balance perceptual quality and fidelity. To achieve minimal distortion while maintaining perfect perceptual quality, Posterior-Mean Rectified Flow (PMRF) proposes a flow based approach where source distribution is minimum distortion estimations. Although PMRF is shown to be effective, its pixel-space modeling approach limits its ability to align with human perception, where human perception is defined as how humans distinguish between two image distributions. In this work, we propose Latent-PMRF, which reformulates PMRF in the latent space of a variational autoencoder (VAE), facilitating better alignment with human perception during optimization. By defining the source distribution on latent representations of minimum distortion estimation, we bound the minimum distortion by the VAE's reconstruction error. Moreover, we reveal the design of VAE is crucial, and our proposed VAE significantly outperforms existing VAEs in both reconstruction and restoration. Extensive experiments on blind face restoration demonstrate the superiority of Latent-PMRF, offering an improved PD-tradeoff compared to existing methods, along with remarkable convergence efficiency, achieving a 5.79X speedup over PMRF in terms of FID. Our code will be available as open-source.

Related papers

• LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter [52.93785843453579]
  Blind face restoration from low-quality (LQ) images is a challenging task that requires high-fidelity image reconstruction and the preservation of facial identity.<n>We propose LAFR, a novel codebook-based latent space adapter that aligns the latent distribution of LQ images with that of HQ counterparts.<n>We show that lightweight finetuning of diffusion prior on just 0.9% of FFHQ dataset is sufficient to achieve results comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-29T14:11:16Z)
• Traversing Distortion-Perception Tradeoff using a Single Score-Based Generative Model [35.91741991271154]
  distortion-perception tradeoff reveals a fundamental conflict between distortion metrics and perceptual quality.<n>We show that a single score network can effectively and flexibly traverse the DP tradeoff for general denoising problems.
  arXiv  Detail & Related papers  (2025-03-26T07:37:53Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration [34.50287066865267]
  Posterior-Mean Rectified Flow (PMRF) is a simple yet highly effective algorithm that approximates this optimal estimator.<n>We investigate the theoretical utility of PMRF and demonstrate that it consistently outperforms previous methods on a variety of image restoration tasks.
  arXiv  Detail & Related papers  (2024-10-01T05:54:07Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Amortized Posterior Sampling with Diffusion Prior Distillation [55.03585818289934]
  Amortized Posterior Sampling is a novel variational inference approach for efficient posterior sampling in inverse problems.<n>Our method trains a conditional flow model to minimize the divergence between the variational distribution and the posterior distribution implicitly defined by the diffusion model.<n>Unlike existing methods, our approach is unsupervised, requires no paired training data, and is applicable to both Euclidean and non-Euclidean domains.
  arXiv  Detail & Related papers  (2024-07-25T09:53:12Z)
• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
  We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences.<n>To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model.<n>Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
  arXiv  Detail & Related papers  (2024-05-26T05:38:50Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• A Theory of the Distortion-Perception Tradeoff in Wasserstein Space [35.25746003630763]
  lower the distortion of an estimator, the more the distribution of its outputs deviates from the distribution of the signals it attempts to estimate. This phenomenon has captured significant attention in image restoration, where it implies that fidelity to ground truth images comes at the expense of perceptual quality. We show how estimators can be constructed from the estimators at the two extremes of the perception-distortion tradeoff.
  arXiv  Detail & Related papers  (2021-07-06T11:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.