Groot: Adversarial Testing for Generative Text-to-Image Models with Tree-based Semantic Transformation

• URL: http://arxiv.org/abs/2402.12100v1
• Date: Mon, 19 Feb 2024 12:31:56 GMT
• Title: Groot: Adversarial Testing for Generative Text-to-Image Models with Tree-based Semantic Transformation
• Authors: Yi Liu, Guowei Yang, Gelei Deng, Feiyue Chen, Yuqi Chen, Ling Shi, Tianwei Zhang, and Yang Liu
• Abstract summary: adversarial testing techniques have been developed to probe whether such models can be prompted to produce Not-Safe-For-Work (NSFW) content. We introduce Groot, the first automated framework leveraging tree-based semantic transformation for adversarial testing of text-to-image models. Our comprehensive evaluation confirms the efficacy of Groot, which not only exceeds the performance of current state-of-the-art approaches but also achieves a remarkable success rate (93.66%) on leading text-to-image models.
• Score: 16.79414725225863
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the prevalence of text-to-image generative models, their safety becomes a critical concern. adversarial testing techniques have been developed to probe whether such models can be prompted to produce Not-Safe-For-Work (NSFW) content. However, existing solutions face several challenges, including low success rate and inefficiency. We introduce Groot, the first automated framework leveraging tree-based semantic transformation for adversarial testing of text-to-image models. Groot employs semantic decomposition and sensitive element drowning strategies in conjunction with LLMs to systematically refine adversarial prompts. Our comprehensive evaluation confirms the efficacy of Groot, which not only exceeds the performance of current state-of-the-art approaches but also achieves a remarkable success rate (93.66%) on leading text-to-image models such as DALL-E 3 and Midjourney.

Related papers

• ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning [62.61187785810336]
  ImageScope is a training-free, three-stage framework that unifies language-guided image retrieval tasks. In the first stage, we improve the robustness of the framework by synthesizing search intents across varying levels of semantic granularity. In the second and third stages, we reflect on retrieval results by verifying predicate propositions locally, and performing pairwise evaluations globally.
  arXiv  Detail & Related papers  (2025-03-13T08:43:24Z)
• Unified Prompt Attack Against Text-to-Image Generation Models [30.24530622359188]
  We propose UPAM, a framework to evaluate the robustness of T2I models from an attack perspective. UPAM unifies the attack on both textual and visual defenses. It also enables gradient-based optimization, overcoming reliance on enumeration for improved efficiency and effectiveness.
  arXiv  Detail & Related papers  (2025-02-23T03:36:18Z)
• DILLEMA: Diffusion and Large Language Models for Multi-Modal Augmentation [0.13124513975412253]
  We present a novel framework for testing vision neural networks that leverages Large Language Models and control-conditioned Diffusion Models. Our approach begins by translating images into detailed textual descriptions using a captioning model. These descriptions are then used to produce new test images through a text-to-image diffusion process.
  arXiv  Detail & Related papers  (2025-02-05T16:35:42Z)
• Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step [77.86514804787622]
  Chain-of-Thought (CoT) reasoning has been extensively explored in large models to tackle complex understanding tasks. We provide the first comprehensive investigation of the potential of CoT reasoning to enhance autoregressive image generation. We propose the Potential Assessment Reward Model (PARM) and PARM++, specialized for autoregressive image generation.
  arXiv  Detail & Related papers  (2025-01-23T18:59:43Z)
• An indicator for effectiveness of text-to-image guardrails utilizing the Single-Turn Crescendo Attack (STCA) [0.0]
  Single-Turn Crescendo Attack (STCA) is an innovative method designed to bypass the ethical safeguards of text-to-text AI models. This study provides a framework for researchers to rigorously evaluate the robustness of guardrails in text-to-image models.
  arXiv  Detail & Related papers  (2024-11-27T19:09:16Z)
• SteerDiff: Steering towards Safe Text-to-Image Diffusion Models [5.781285400461636]
  Text-to-image (T2I) diffusion models can be misused to produce inappropriate content. We introduce SteerDiff, a lightweight adaptor module designed to act as an intermediary between user input and the diffusion model. We conduct extensive experiments across various concept unlearning tasks to evaluate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-03T17:34:55Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Unified Text-to-Image Generation and Retrieval [96.72318842152148]
  We propose a unified framework in the context of Multimodal Large Language Models (MLLMs) We first explore the intrinsic discrimi abilities of MLLMs and introduce a generative retrieval method to perform retrieval in a training-free manner. We then unify generation and retrieval in an autoregressive generation way and propose an autonomous decision module to choose the best-matched one between generated and retrieved images.
  arXiv  Detail & Related papers  (2024-06-09T15:00:28Z)
• ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users [18.3621509910395]
  We propose a novel Automatic Red-Teaming framework, ART, to evaluate the safety risks of text-to-image models. With our comprehensive experiments, we reveal the toxicity of the popular open-source text-to-image models. We also introduce three large-scale red-teaming datasets for studying the safety risks associated with text-to-image models.
  arXiv  Detail & Related papers  (2024-05-24T07:44:27Z)
• Counterfactual Image Generation for adversarially robust and interpretable Classifiers [1.3859669037499769]
  We propose a unified framework leveraging image-to-image translation Generative Adrial Networks (GANs) to produce counterfactual samples. This is achieved by combining the classifier and discriminator into a single model that attributes real images to their respective classes and flags generated images as "fake" We show how the model exhibits improved robustness to adversarial attacks, and we show how the discriminator's "fakeness" value serves as an uncertainty measure of the predictions.
  arXiv  Detail & Related papers  (2023-10-01T18:50:29Z)
• SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution [21.93748586123046]
  We develop and exhibit the first prompt attacks on Midjourney, resulting in the production of abundant NSFW images. Our framework, SurrogatePrompt, systematically generates attack prompts, utilizing large language models, image-to-text, and image-to-image modules. Results disclose an 88% success rate in bypassing Midjourney's proprietary safety filter with our attack prompts.
  arXiv  Detail & Related papers  (2023-09-25T13:20:15Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models [54.19289900203071]
  The rise in popularity of text-to-image generative artificial intelligence has attracted widespread public interest. We demonstrate that this technology can be attacked to generate content that subtly manipulates its users. We propose a Backdoor Attack on text-to-image Generative Models (BAGM) Our attack is the first to target three popular text-to-image generative models across three stages of the generative process.
  arXiv  Detail & Related papers  (2023-07-31T08:34:24Z)
• Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models [55.04969603431266]
  This paper proposes a method for generating images of customized objects specified by users. The method is based on a general framework that bypasses the lengthy optimization required by previous approaches. We demonstrate through experiments that our proposed method is able to synthesize images with compelling output quality, appearance diversity, and object fidelity.
  arXiv  Detail & Related papers  (2023-04-05T17:59:32Z)
• GR-GAN: Gradual Refinement Text-to-image Generation [15.99543073122574]
  This paper proposes a Gradual Refinement Generative Adversarial Network (GR-GAN) to alleviate the problem efficiently. A GRG module is designed to generate images from low resolution to high resolution with the corresponding text constraints. A ITM module is designed to provide image-text matching losses at both sentence-image level and word-region level.
  arXiv  Detail & Related papers  (2022-05-23T12:42:04Z)
• Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models [86.02610674750345]
  Adversarial GLUE (AdvGLUE) is a new multi-task benchmark to explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks. We apply 14 adversarial attack methods to GLUE tasks to construct AdvGLUE, which is further validated by humans for reliable annotations. All the language models and robust training methods we tested perform poorly on AdvGLUE, with scores lagging far behind the benign accuracy.
  arXiv  Detail & Related papers  (2021-11-04T12:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.