PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization

• URL: http://arxiv.org/abs/2511.22119v1
• Date: Thu, 27 Nov 2025 05:22:10 GMT
• Title: PROMPTMINER: Black-Box Prompt Stealing against Text-to-Image Generative Models via Reinforcement Learning and Fuzz Optimization
• Authors: Mingzhe Li, Renhao Zhang, Zhiyang Wen, Siqi Pan, Bruno Castro da Silva, Juan Zhai, Shiqing Ma,
• Abstract summary: Text-to-image (T2I) generative models can synthesize realistic, high-quality images directly from textual prompts.<n>Prompt stealing attacks enable unauthorized extraction and reuse of carefully engineered prompts.<n>We propose PROMPTMINER, a black-box prompt stealing framework that decouples the task into two phases.<n>ProMPTMINER achieves superior results, with CLIP similarity up to 0.958 and textual alignment with SBERT up to 0.751, surpassing all baselines.
• Score: 27.094857962888288
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-image (T2I) generative models such as Stable Diffusion and FLUX can synthesize realistic, high-quality images directly from textual prompts. The resulting image quality depends critically on well-crafted prompts that specify both subjects and stylistic modifiers, which have become valuable digital assets. However, the rising value and ubiquity of high-quality prompts expose them to security and intellectual-property risks. One key threat is the prompt stealing attack, i.e., the task of recovering the textual prompt that generated a given image. Prompt stealing enables unauthorized extraction and reuse of carefully engineered prompts, yet it can also support beneficial applications such as data attribution, model provenance analysis, and watermarking validation. Existing approaches often assume white-box gradient access, require large-scale labeled datasets for supervised training, or rely solely on captioning without explicit optimization, limiting their practicality and adaptability. To address these challenges, we propose PROMPTMINER, a black-box prompt stealing framework that decouples the task into two phases: (1) a reinforcement learning-based optimization phase to reconstruct the primary subject, and (2) a fuzzing-driven search phase to recover stylistic modifiers. Experiments across multiple datasets and diffusion backbones demonstrate that PROMPTMINER achieves superior results, with CLIP similarity up to 0.958 and textual alignment with SBERT up to 0.751, surpassing all baselines. Even when applied to in-the-wild images with unknown generators, it outperforms the strongest baseline by 7.5 percent in CLIP similarity, demonstrating better generalization. Finally, PROMPTMINER maintains strong performance under defensive perturbations, highlighting remarkable robustness. Code: https://github.com/aaFrostnova/PromptMiner

Related papers

• RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration [94.49712266736141]
  RAM++ is a two-stage framework for all-in-one image restoration.<n>It integrates high-level semantic understanding with low-level texture generation.<n>It addresses the limitations of existing degradation-oriented methods in extreme scenarios.
  arXiv  Detail & Related papers  (2025-09-15T15:24:15Z)
• GhostPrompt: Jailbreaking Text-to-image Generative Models based on Dynamic Optimization [19.44247617251449]
  We introduce GhostPrompt, the first automated jailbreak framework that combines dynamic prompt optimization with multimodal feedback.<n>It achieves state-of-the-art performance, increasing the ShieldLM-7B bypass rate from 12.5% to 99.0%.<n>It generalizes to unseen filters including GPT-4.1 and successfully jailbreaks DALLE 3 to generate NSFW images.
  arXiv  Detail & Related papers  (2025-05-25T05:13:06Z)
• Lumina-Image 2.0: A Unified and Efficient Image Generative Framework [76.44331001702379]
  Lumina-Image 2.0 is a text-to-image generation framework that achieves significant progress compared to previous work.<n>It adopts a unified architecture (Unified Next-DiT) that treats text and image tokens as a joint sequence.<n>We introduce a unified captioning system, Unified Captioner (UniCap), specifically designed for T2I generation tasks.
  arXiv  Detail & Related papers  (2025-03-27T17:57:07Z)
• FRAP: Faithful and Realistic Text-to-Image Generation with Adaptive Prompt Weighting [18.708185548091716]
  FRAP is a simple, yet effective approach based on adaptively adjusting the per-token prompt weights.<n>We show FRAP generates images with significantly higher prompt-image alignment to prompts from complex datasets.<n>We also explore combining FRAP with prompt rewriting LLM to recover their degraded prompt-image alignment.
  arXiv  Detail & Related papers  (2024-08-21T15:30:35Z)
• Improving Text-to-Image Consistency via Automatic Prompt Optimization [26.2587505265501]
  We introduce a T2I optimization-by-prompting framework, OPT2I, to improve prompt-image consistency in T2I models. Our framework starts from a user prompt and iteratively generates revised prompts with the goal of maximizing a consistency score.
  arXiv  Detail & Related papers  (2024-03-26T15:42:01Z)
• Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers [120.49126407479717]
  This paper explores text-to-image diffusion models for Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) We highlight a pivotal discovery: the capacity of text-to-image diffusion models to seamlessly bridge the gap between sketches and photos.
  arXiv  Detail & Related papers  (2024-03-12T00:02:03Z)
• Universal Prompt Optimizer for Safe Text-to-Image Generation [27.32589928097192]
  We propose the first universal prompt for safe T2I (POSI) generation in black-box scenario.<n>Our approach can effectively reduce the likelihood of various T2I models in generating inappropriate images.
  arXiv  Detail & Related papers  (2024-02-16T18:36:36Z)
• Sentence-level Prompts Benefit Composed Image Retrieval [69.78119883060006]
  Composed image retrieval (CIR) is the task of retrieving specific images by using a query that involves both a reference image and a relative caption. We propose to leverage pretrained V-L models, e.g., BLIP-2, to generate sentence-level prompts. Our proposed method performs favorably against the state-of-the-art CIR methods on the Fashion-IQ and CIRR datasets.
  arXiv  Detail & Related papers  (2023-10-09T07:31:44Z)
• Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval [29.838375158101027]
  Contrastive language-image pre-training (CLIP) has demonstrated remarkable zero-shot classification ability. We propose X-MoRe, a novel inference method comprising two key steps: (1) cross-modal retrieval and (2) modal-confidence-based ensemble. X-MoRe demonstrates robust performance across a diverse set of tasks without the need for additional training.
  arXiv  Detail & Related papers  (2023-08-29T13:02:35Z)
• Planting a SEED of Vision in Large Language Model [73.17530130368053]
  We present SEED, an elaborate image tokenizer that empowers Large Language Models (LLMs) with the ability to SEE and Draw at the same time. This version of SEED was trained in 5.7 days using only 64 V100 GPUs and 5M publicly available image-text pairs.
  arXiv  Detail & Related papers  (2023-07-16T13:41:39Z)
• PV2TEA: Patching Visual Modality to Textual-Established Information Extraction [59.76117533540496]
  We patch the visual modality to the textual-established attribute information extractor. PV2TEA is an encoder-decoder architecture equipped with three bias reduction schemes. Empirical results on real-world e-Commerce datasets demonstrate up to 11.74% absolute (20.97% relatively) F1 increase over unimodal baselines.
  arXiv  Detail & Related papers  (2023-06-01T05:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.