From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis

• URL: http://arxiv.org/abs/2504.01395v1
• Date: Wed, 02 Apr 2025 06:30:55 GMT
• Title: From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis
• Authors: Kecen Li, Chen Gong, Xiaochen Li, Yuzhong Zhao, Xinwen Hou, Tianhao Wang,
• Abstract summary: Differentially private (DP) image synthesis aims to generate synthetic images from a sensitive dataset.<n>We propose a two-stage DP image synthesis framework, where diffusion models learn to generate synthetic images from easy to hard.<n>We conduct experiments to present that on the average of four investigated image datasets, the fidelity and utility metrics of our synthetic images are 33.1% and 2.1% better than the state-of-the-art method.
• Score: 14.93795597224185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differentially private (DP) image synthesis aims to generate synthetic images from a sensitive dataset, alleviating the privacy leakage concerns of organizations sharing and utilizing synthetic images. Although previous methods have significantly progressed, especially in training diffusion models on sensitive images with DP Stochastic Gradient Descent (DP-SGD), they still suffer from unsatisfactory performance. In this work, inspired by curriculum learning, we propose a two-stage DP image synthesis framework, where diffusion models learn to generate DP synthetic images from easy to hard. Unlike existing methods that directly use DP-SGD to train diffusion models, we propose an easy stage in the beginning, where diffusion models learn simple features of the sensitive images. To facilitate this easy stage, we propose to use `central images', simply aggregations of random samples of the sensitive dataset. Intuitively, although those central images do not show details, they demonstrate useful characteristics of all images and only incur minimal privacy costs, thus helping early-phase model training. We conduct experiments to present that on the average of four investigated image datasets, the fidelity and utility metrics of our synthetic images are 33.1% and 2.1% better than the state-of-the-art method.

Related papers

• Diffusion Curriculum: Synthetic-to-Real Generative Curriculum Learning via Image-Guided Diffusion [19.54496184675988]
  Low-quality or scarce data has posed significant challenges for training deep neural networks in practice. Diffusion Curriculum (DisCL) adjusts the image guidance level of image synthesis for each training stage. DisCL focuses on lower-guidance images of high-quality to learn features as a warm-up of learning higher-guidance images that might be weak on diversity or quality.
  arXiv  Detail & Related papers  (2024-10-17T15:33:35Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis [65.7968515029306]
  We propose a novel Coarse-to-Fine Latent Diffusion (CFLD) method for Pose-Guided Person Image Synthesis (PGPIS) A perception-refined decoder is designed to progressively refine a set of learnable queries and extract semantic understanding of person images as a coarse-grained prompt.
  arXiv  Detail & Related papers  (2024-02-28T06:07:07Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining [13.823621924706348]
  Differential Privacy (DP) image data synthesis allows organizations to share and utilize synthetic images without privacy concerns. Previous methods incorporate the advanced techniques of generative models and pre-training on a public dataset to produce exceptional DP image data. This paper proposes a novel DP image synthesis method, termed PRIVIMAGE, which meticulously selects pre-training data.
  arXiv  Detail & Related papers  (2023-10-19T14:04:53Z)
• Collaborative Score Distillation for Consistent Visual Synthesis [70.29294250371312]
  Collaborative Score Distillation (CSD) is based on the Stein Variational Gradient Descent (SVGD) We show the effectiveness of CSD in a variety of tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.
  arXiv  Detail & Related papers  (2023-07-04T17:31:50Z)
• Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion models [0.0]
  Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images. We show that segmentation networks trained on synthetic images reach Dice scores that are 80% - 90% of Dice scores when training with real images. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required.
  arXiv  Detail & Related papers  (2023-06-05T15:56:30Z)
• Generalizable Synthetic Image Detection via Language-guided Contrastive Learning [22.533225521726116]
  malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, raises significant concerns regarding the authenticity of images. We propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning. It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models.
  arXiv  Detail & Related papers  (2023-05-23T08:13:27Z)
• Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
  We show how denoising diffusion models can be applied for high-fidelity person image synthesis. Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
  arXiv  Detail & Related papers  (2022-11-22T18:59:50Z)
• Joint Deep Learning of Facial Expression Synthesis and Recognition [97.19528464266824]
  We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER. The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions. In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
  arXiv  Detail & Related papers  (2020-02-06T10:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.