Context-Aware Full Body Anonymization using Text-to-Image Diffusion Models

• URL: http://arxiv.org/abs/2410.08551v2
• Date: Thu, 17 Oct 2024 14:04:01 GMT
• Title: Context-Aware Full Body Anonymization using Text-to-Image Diffusion Models
• Authors: Pascal Zwick, Kevin Roesch, Marvin Klemp, Oliver Bringmann,
• Abstract summary: Anonymization plays a key role in protecting sensible information of individuals in real world datasets. In this paper, we propose a workflow for full body person anonymization utilizing Stable Diffusion as a generative backend. We show that our method outperforms state-of-the art anonymization pipelines with respect to image quality, resolution, Inception Score (IS) and Frechet Inception Distance (FID)
• Score: 1.5088726951324294
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anonymization plays a key role in protecting sensible information of individuals in real world datasets. Self-driving cars for example need high resolution facial features to track people and their viewing direction to predict future behaviour and react accordingly. In order to protect people's privacy whilst keeping important features in the dataset, it is important to replace the full body of a person with a highly detailed anonymized one. In contrast to doing face anonymization, full body replacement decreases the ability of recognizing people by their hairstyle or clothes. In this paper, we propose a workflow for full body person anonymization utilizing Stable Diffusion as a generative backend. Text-to-image diffusion models, like Stable Diffusion, OpenAI's DALL-E or Midjourney, have become very popular in recent time, being able to create photorealistic images from a single text prompt. We show that our method outperforms state-of-the art anonymization pipelines with respect to image quality, resolution, Inception Score (IS) and Frechet Inception Distance (FID). Additionally, our method is invariant with respect to the image generator and thus able to be used with the latest models available.

Related papers

• FAKER: Full-body Anonymization with Human Keypoint Extraction for Real-time Video Deidentification [0.0]
  We propose a novel approach that employs a significantly smaller model to achieve real-time full-body anonymization of individuals in videos. By leveraging pose estimation algorithms, our approach accurately represents information regarding individuals' positions, movements, and postures. This algorithm can be seamlessly integrated into CCTV or IP camera systems installed in various industrial settings.
  arXiv  Detail & Related papers  (2024-08-06T04:59:23Z)
• Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation [56.46932751058042]
  We train a learnable prompt prefix for text-to-image diffusion models, which forces the model to generate anonymized facial identities. Experiments demonstrate the successful anonymization performance of APL, which anonymizes any specific individuals without compromising the quality of non-identity-specific image generation.
  arXiv  Detail & Related papers  (2024-05-27T07:38:26Z)
• Privacy-preserving Optics for Enhancing Protection in Face De-identification [60.110274007388135]
  We propose a hardware-level face de-identification method to solve this vulnerability. We also propose an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input.
  arXiv  Detail & Related papers  (2024-03-31T19:28:04Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• Attribute-preserving Face Dataset Anonymization via Latent Code Optimization [64.4569739006591]
  We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN. We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
  arXiv  Detail & Related papers  (2023-03-20T17:34:05Z)
• DeepPrivacy2: Towards Realistic Full-Body Anonymization [0.0]
  We propose a novel anonymization framework (DeepPrivacy2) for realistic anonymization of human figures and faces. We introduce a new large and diverse dataset for human figure synthesis, which significantly improves image quality and diversity of generated images.
  arXiv  Detail & Related papers  (2022-11-17T10:52:27Z)
• Learnable Privacy-Preserving Anonymization for Pedestrian Images [27.178354411900127]
  This paper studies a novel privacy-preserving anonymization problem for pedestrian images. It preserves personal identity information (PII) for authorized models and prevents PII from being recognized by third parties. We propose a joint learning reversible anonymization framework, which can reversibly generate full-body anonymous images.
  arXiv  Detail & Related papers  (2022-07-24T07:04:16Z)
• OPOM: Customized Invisible Cloak towards Face Privacy Protection [58.07786010689529]
  We investigate the face privacy protection from a technology standpoint based on a new type of customized cloak. We propose a new method, named one person one mask (OPOM), to generate person-specific (class-wise) universal masks. The effectiveness of the proposed method is evaluated on both common and celebrity datasets.
  arXiv  Detail & Related papers  (2022-05-24T11:29:37Z)
• Realistic Full-Body Anonymization with Surface-Guided GANs [7.37907896341367]
  We propose a new anonymization method that generates realistic humans for in-the-wild images. A key part of our design is to guide adversarial nets by dense pixel-to-surface correspondences between an image and a canonical 3D surface. We demonstrate that surface guidance significantly improves image quality and diversity of samples, yielding a highly practical generator.
  arXiv  Detail & Related papers  (2022-01-06T18:57:59Z)
• CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks [12.20367903755194]
  CIAGAN is a model for image and video anonymization based on conditional generative adversarial networks. Our model is able to remove the identifying characteristics of faces and bodies while producing high-quality images and videos.
  arXiv  Detail & Related papers  (2020-05-19T15:56:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.