Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling

• URL: http://arxiv.org/abs/2602.23262v1
• Date: Thu, 26 Feb 2026 17:36:48 GMT
• Title: Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling
• Authors: Jasmine Bayrooti, Weiwei Kong, Natalia Ponomareva, Carlos Esteves, Ameesh Makadia, Amanda Prorok,
• Abstract summary: We propose a spectral DP framework based on the hypothesis that the most privacy-sensitive portions of an image are often low-frequency components in the wavelet space.<n>By restricting the privacy budget to the global structures of the image in the first stage, and leveraging the post-processing property of DP for detail refinement, we achieve promising trade-offs between privacy and utility.
• Score: 25.30412582520304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative models trained on sensitive image datasets risk memorizing and reproducing individual training examples, making strong privacy guarantees essential. While differential privacy (DP) provides a principled framework for such guarantees, standard DP finetuning (e.g., with DP-SGD) often results in severe degradation of image quality, particularly in high-frequency textures, due to the indiscriminate addition of noise across all model parameters. In this work, we propose a spectral DP framework based on the hypothesis that the most privacy-sensitive portions of an image are often low-frequency components in the wavelet space (e.g., facial features and object shapes) while high-frequency components are largely generic and public. Based on this hypothesis, we propose the following two-stage framework for DP image generation with coarse image intermediaries: (1) DP finetune an autoregressive spectral image tokenizer model on the low-resolution wavelet coefficients of the sensitive images, and (2) perform high-resolution upsampling using a publicly pretrained super-resolution model. By restricting the privacy budget to the global structures of the image in the first stage, and leveraging the post-processing property of DP for detail refinement, we achieve promising trade-offs between privacy and utility. Experiments on the MS-COCO and MM-CelebA-HQ datasets show that our method generates images with improved quality and style capture relative to other leading DP image frameworks.

Related papers

• DPImageBench: A Unified Benchmark for Differentially Private Image Synthesis [18.158096061348157]
  Differentially private (DP) image synthesis aims to generate artificial images that retain the properties of sensitive images while protecting the privacy of individual images within the dataset.<n>Despite recent advancements, inconsistent--and sometimes flawed--evaluation protocols have been applied across studies.<n>This paper introduces DPImageBench for DP image synthesis, with thoughtful design across several dimensions.
  arXiv  Detail & Related papers  (2025-03-18T19:37:35Z)
• MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorize [18.73205699076486]
  We introduce a diffusion framework leveraging multi-scale latent factorization.<n>Our framework decomposes the denoising target, typically latent features from a pretrained Variational Autoencoder, into a low-frequency base signal.<n>Our proposed architecture facilitates reduced sampling steps during the residual learning stage.
  arXiv  Detail & Related papers  (2025-01-23T03:18:23Z)
• Self-supervised Deep Hyperspectral Inpainting with the Plug and Play and Deep Image Prior Models [6.8557067473167415]
  Hyperspectral images are typically composed of hundreds of narrow and contiguous spectral bands, each containing information regarding the composition of the imaged scene.<n>These images can be affected by various sources of noise, distortions data, or material loss, which can significantly degrade their quality and usefulness.<n>This paper introduces a converge a guaranteed algorithm, LRS-nt-DIP, which successfully addresses the instability issue of DHP.
  arXiv  Detail & Related papers  (2025-01-14T15:18:28Z)
• Differentially Private Adaptation of Diffusion Models via Noisy Aggregated Embeddings [23.687702204151872]
  Textual Inversion (TI) learns an embedding vector for an image or set of images, to enable adaptation under differential privacy constraints.<n>We show DPAgg-TI outperforms DP-SGD finetuning in both utility and robustness under the same privacy budget.
  arXiv  Detail & Related papers  (2024-11-22T00:09:49Z)
• High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity [69.32473738284374]
  Diffusion models have revolutionized text-to-image synthesis by delivering exceptional quality, fine detail resolution, and strong contextual awareness.<n>We propose DiffDIS, a diffusion-driven segmentation model that taps into the potential of the pre-trained U-Net within diffusion models.<n>Experiments on the DIS5K dataset demonstrate the superiority of DiffDIS, achieving state-of-the-art results through a streamlined inference process.
  arXiv  Detail & Related papers  (2024-10-14T02:49:23Z)
• QMambaBSR: Burst Image Super-Resolution with Query State Space Model [55.56075874424194]
  Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames.<n>In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance.<n>We introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp)
  arXiv  Detail & Related papers  (2024-08-16T11:15:29Z)
• DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning [38.697798191642136]
  We develop the first differentially private (DP) retrieval-augmented generation algorithm. It is capable of generating high-quality image samples while providing provable privacy guarantees.
  arXiv  Detail & Related papers  (2024-03-21T14:17:28Z)
• Generative Diffusion Prior for Unified Image Restoration and Enhancement [62.76390152617949]
  Existing image restoration methods mostly leverage the posterior distribution of natural images. We propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems.
  arXiv  Detail & Related papers  (2023-04-03T16:52:43Z)
• CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution [91.56337748920662]
  Diffusion probabilistic models (DPM) have been widely adopted in image-to-image translation. We propose a simple but non-trivial DPM-based super-resolution post-process framework,i.e., cDPMSR. Our method surpasses prior attempts on both qualitative and quantitative results.
  arXiv  Detail & Related papers  (2023-02-14T15:13:33Z)
• Universal Face Restoration With Memorized Modulation [73.34750780570909]
  This paper proposes a Restoration with Memorized Modulation (RMM) framework for universal Blind Face Restoration (BFR) We apply random noise as well as unsupervised wavelet memory to adaptively modulate the face-enhancement generator. Experimental results show the superiority of the proposed method compared with the state-of-the-art methods, and a good generalization in the wild.
  arXiv  Detail & Related papers  (2021-10-03T15:55:07Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.