Towards Safe Synthetic Image Generation On the Web: A Multimodal Robust NSFW Defense and Million Scale Dataset

• URL: http://arxiv.org/abs/2504.11707v1
• Date: Wed, 16 Apr 2025 02:10:42 GMT
• Title: Towards Safe Synthetic Image Generation On the Web: A Multimodal Robust NSFW Defense and Million Scale Dataset
• Authors: Muhammad Shahid Muneer, Simon S. Woo,
• Abstract summary: A multimodal defense is developed to distinguish safe and NSFW text and images.<n>Our model performs well against existing SOTA NSFW detection methods in terms of accuracy and recall.
• Score: 20.758637391023345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the past years, we have witnessed the remarkable success of Text-to-Image (T2I) models and their widespread use on the web. Extensive research in making T2I models produce hyper-realistic images has led to new concerns, such as generating Not-Safe-For-Work (NSFW) web content and polluting the web society. To help prevent misuse of T2I models and create a safer web environment for users features like NSFW filters and post-hoc security checks are used in these models. However, recent work unveiled how these methods can easily fail to prevent misuse. In particular, adversarial attacks on text and image modalities can easily outplay defensive measures. %Exploiting such leads to the growing concern of preventing adversarial attacks on text and image modalities. Moreover, there is currently no robust multimodal NSFW dataset that includes both prompt and image pairs and adversarial examples. This work proposes a million-scale prompt and image dataset generated using open-source diffusion models. Second, we develop a multimodal defense to distinguish safe and NSFW text and images, which is robust against adversarial attacks and directly alleviates current challenges. Our extensive experiments show that our model performs well against existing SOTA NSFW detection methods in terms of accuracy and recall, drastically reducing the Attack Success Rate (ASR) in multimodal adversarial attack scenarios. Code: https://github.com/shahidmuneer/multimodal-nsfw-defense.

Related papers

• Clean Image May be Dangerous: Data Poisoning Attacks Against Deep Hashing [71.30876587855867]
  We show that even clean query images can be dangerous, inducing malicious target retrieval results, like undesired or illegal images. Specifically, we first train a surrogate model to simulate the behavior of the target deep hashing model. Then, a strict gradient matching strategy is proposed to generate the poisoned images.
  arXiv  Detail & Related papers  (2025-03-27T07:54:27Z)
• CROPS: Model-Agnostic Training-Free Framework for Safe Image Synthesis with Latent Diffusion Models [13.799517170191919]
  Recent research has shown that safety checkers have vulnerabilities against adversarial attacks, allowing them to generate Not Safe For Work (NSFW) images.<n>We propose CROPS, a model-agnostic framework that easily defends against adversarial attacks generating NSFW images without requiring additional training.
  arXiv  Detail & Related papers  (2025-01-09T16:43:21Z)
• AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models [39.11841245506388]
  Malicious users often exploit text-to-image (T2I) models to generate Not-Safe-for-Work (NSFW) images.<n>We introduce AEIOU, a framework that is Adaptable, Efficient, Interpretable, Optimizable, and Unified against NSFW prompts in T2I models.
  arXiv  Detail & Related papers  (2024-12-24T03:17:45Z)
• Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vector [97.92369017531038]
  We build a new laRge-scale Adervsarial images dataset with Diverse hArmful Responses (RADAR) We then develop a novel iN-time Embedding-based AdveRSarial Image DEtection (NEARSIDE) method, which exploits a single vector that distilled from the hidden states of Visual Language Models (VLMs) to achieve the detection of adversarial images against benign ones in the input.
  arXiv  Detail & Related papers  (2024-10-30T10:33:10Z)
• AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion models [20.37481116837779]
  AdvI2I is a novel framework that manipulates input images to induce diffusion models to generate NSFW content. By optimizing a generator to craft adversarial images, AdvI2I circumvents existing defense mechanisms. We show that both AdvI2I and AdvI2I-Adaptive can effectively bypass current safeguards.
  arXiv  Detail & Related papers  (2024-10-28T19:15:06Z)
• Multimodal Pragmatic Jailbreak on Text-to-image Models [43.67831238116829]
  This work introduces a novel type of jailbreak, which triggers T2I models to generate the image with visual text. We benchmark nine representative T2I models, including two close-source commercial models. All tested models suffer from such type of jailbreak, with rates of unsafe generation ranging from 8% to 74%.
  arXiv  Detail & Related papers  (2024-09-27T21:23:46Z)
• On the Multi-modal Vulnerability of Diffusion Models [56.08923332178462]
  We propose MMP-Attack to manipulate the generation results of diffusion models by appending a specific suffix to the original prompt.<n>Our goal is to induce diffusion models to generate a specific object while simultaneously eliminating the original object.
  arXiv  Detail & Related papers  (2024-02-02T12:39:49Z)
• DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection [64.77548539959501]
  DiffProtect produces more natural-looking encrypted images than state-of-the-art methods. It achieves significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.
  arXiv  Detail & Related papers  (2023-05-23T02:45:49Z)
• SneakyPrompt: Jailbreaking Text-to-image Generative Models [20.645304189835944]
  We propose SneakyPrompt, the first automated attack framework, to jailbreak text-to-image generative models. Given a prompt that is blocked by a safety filter, SneakyPrompt repeatedly queries the text-to-image generative model and strategically perturbs tokens in the prompt based on the query results to bypass the safety filter. Our evaluation shows that SneakyPrompt not only successfully generates NSFW images, but also outperforms existing text adversarial attacks when extended to jailbreak text-to-image generative models.
  arXiv  Detail & Related papers  (2023-05-20T03:41:45Z)
• Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains [80.11169390071869]
  Adversarial examples have posed a severe threat to deep neural networks due to their transferable nature. We propose a Beyond ImageNet Attack (BIA) to investigate the transferability towards black-box domains. Our methods outperform state-of-the-art approaches by up to 7.71% (towards coarse-grained domains) and 25.91% (towards fine-grained domains) on average.
  arXiv  Detail & Related papers  (2022-01-27T14:04:27Z)
• Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks [154.31827097264264]
  Adversarial training is a popular defense strategy against attack threat models with bounded Lp norms. We propose Dual Manifold Adversarial Training (DMAT) where adversarial perturbations in both latent and image spaces are used in robustifying the model. Our DMAT improves performance on normal images, and achieves comparable robustness to the standard adversarial training against Lp attacks.
  arXiv  Detail & Related papers  (2020-09-05T06:00:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.