Mental Damage: Caption Poisoning Attacks on Retrieval-Augmented Text-to-Music Generation

• URL: http://arxiv.org/abs/2605.30365v1
• Date: Mon, 18 May 2026 02:11:57 GMT
• Title: Mental Damage: Caption Poisoning Attacks on Retrieval-Augmented Text-to-Music Generation
• Abstract summary: Retrieval-augmented text-to-music (TTM) systems augment underspecified user prompts using captions retrieved from a music caption dataset.<n>We show that an attacker can poison the database by injecting a small number of crafted music captions.<n>To achieve the music caption poisoning attack, we propose a dual-layer caption poisoning strategy.
• Score: 10.724986873079827
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented text-to-music (TTM) systems augment underspecified user prompts using captions retrieved from a music caption dataset. This design introduces an integrity dependency on the music knowledge database. We show that an attacker can poison the database by injecting a small number of crafted music captions, causing the system to retrieve malicious captions that bias prompt augmentation and steer generation away from the user's intended function, without modifying the user prompt, retriever, or generator. To achieve the music caption poisoning attack, we propose a dual-layer caption poisoning strategy that preserves high-level retrieval anchors while injecting low-level acoustic descriptors to steer prompt augmentation and downstream music generation toward an attacker-chosen target intent. In a MusicCaps knowledge database, CLAP retriever, and MusicGen pipeline, poisoned generations move substantially closer to the attacker's target, while remaining comparably aligned with the original user query. These results expose a practical integrity risk for retrieval-augmented creative AI systems. Our demo can be found at: https://yizhu-wen.github.io/Mental-Damage/

Related papers

• ViTeGate: Visual-Textual Triggered Knowledge Poisoning for Vision-Language Retrieval-Augmented Generation [10.286163249620545]
  ViTeGate is a visual-textual triggered knowledge poisoning attack for Vision-Language Retrieval-Augmented Generation (VLRAG) systems.<n>By coordinating retrieval and generation, ViTeGate reduces poison exposure when the visual trigger is absent and preserves normal responses when the textual trigger is absent.<n>On InfoSeek, ViTeGate achieves an attack success rate of up to 0.98 while maintaining a clean answer accuracy of up to 0.93.
  arXiv  Detail & Related papers  (2026-09-13T17:39:37Z)
• Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation [47.04195212078377]
  Generative AI systems for music and video commonly use text-based filters to prevent the regurgitation of copyrighted material.<n>We introduce Adversarial PhoneTic Prompting (APT), a novel attack that bypasses these safeguards by exploiting phonetic memorization.<n>We demonstrate that leading Lyrics-to-Song (L2S) models like SUNO and YuE regenerate songs with striking melodic and rhythmic similarity to their copyrighted originals when prompted with these altered lyrics.
  arXiv  Detail & Related papers  (2025-07-23T21:11:47Z)
• LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users [50.18141341939909]
  We describe a vulnerability in language models trained with user feedback.<n>A single user can persistently alter LM knowledge and behavior.<n>We show that this attack can be used to insert factual knowledge the model did not previously possess.
  arXiv  Detail & Related papers  (2025-07-03T17:55:40Z)
• Double Entendre: Robust Audio-Based AI-Generated Lyrics Detection via Multi-View Fusion [11.060929679400667]
  We propose a multimodal, modular late-fusion pipeline that combines automatically transcribed lyrics and speech features capturing lyrics-related information within the audio.<n>Our method, DE-detect, outperforms existing lyrics-based detectors while also being more robust to audio perturbations.
  arXiv  Detail & Related papers  (2025-06-19T02:56:49Z)
• Traceback of Poisoning Attacks to Retrieval-Augmented Generation [18.902988029537365]
  Research has revealed RAG's susceptibility to poisoning attacks, where the attacker injects poisoned texts into the knowledge database.<n>Existing defenses, which predominantly focus on inference-time mitigation, have proven insufficient against sophisticated attacks.<n>We introduce RAGForensics, the first traceback system for RAG, designed to identify poisoned texts within the knowledge database that are responsible for the attacks.
  arXiv  Detail & Related papers  (2025-04-30T14:10:02Z)
• Corpus Poisoning via Approximate Greedy Gradient Descent [48.5847914481222]
  We propose Approximate Greedy Gradient Descent, a new attack on dense retrieval systems based on the widely used HotFlip method for generating adversarial passages. We show that our method achieves a high attack success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains.
  arXiv  Detail & Related papers  (2024-06-07T17:02:35Z)
• Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggers [46.19574403393449]
  This paper investigates a novel attack scenario where the attackers aim to mislead the retrieval system into retrieving the attacker-specified contents.<n>Those contents, injected into the retrieval corpus by attackers, can include harmful text like hate speech or spam.<n>Unlike prior methods that rely on model weights and generate conspicuous, unnatural outputs, we propose a covert backdoor attack triggered by grammar errors.
  arXiv  Detail & Related papers  (2024-02-21T05:03:07Z)
• PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models [45.409248316497674]
  Large language models (LLMs) have achieved remarkable success due to their exceptional generative capabilities. Retrieval-Augmented Generation (RAG) is a state-of-the-art technique to mitigate these limitations. We find that the knowledge database in a RAG system introduces a new and practical attack surface. Based on this attack surface, we propose PoisonedRAG, the first knowledge corruption attack to RAG.
  arXiv  Detail & Related papers  (2024-02-12T18:28:36Z)
• Hidden Backdoor Attack against Semantic Segmentation Models [60.0327238844584]
  The emphbackdoor attack intends to embed hidden backdoors in deep neural networks (DNNs) by poisoning training data. We propose a novel attack paradigm, the emphfine-grained attack, where we treat the target label from the object-level instead of the image-level. Experiments show that the proposed methods can successfully attack semantic segmentation models by poisoning only a small proportion of training data.
  arXiv  Detail & Related papers  (2021-03-06T05:50:29Z)
• Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
  We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN) We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
  arXiv  Detail & Related papers  (2020-10-28T02:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.