Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation

• URL: http://arxiv.org/abs/2605.30031v1
• Date: Thu, 28 May 2026 14:53:27 GMT
• Title: Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation
• Abstract summary: Large Audio Language Models (LALMs) expand jailbreak risks from token-level prompting to the full speech perception-to-reasoning pipeline.<n>This paper provides a unified taxonomy and a controlled empirical evaluation of LALM jailbreak attacks and defenses.
• Score: 16.72528638767562
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Audio Language Models (LALMs) expand jailbreak risks from token-level prompting to the full speech perception-to-reasoning pipeline, where unsafe behavior can be induced through semantics, acoustic style, signal artifacts, or internal representations. Existing work studies these risks under heterogeneous threat models and evaluation protocols, making it difficult to compare attack practicality or defense utility. This paper provides a unified taxonomy and a controlled empirical evaluation of LALM jailbreak attacks and defenses. We organize prior work into semantic, acoustic, signal, and embedding-layer attacks; guard-based, training-free, and training-based defenses; and cross-modal, audio-native, and interactive benchmarks. We then evaluate representative attacks and defenses across ten open-source LALMs, measuring not only attack success rate but also benign refusal and latency. Our results show that Acoustic Best-of-N reveals strong worst-case audio-space vulnerabilities, Narrative Framing is an effective low-latency semantic threat, and current defenses trade robustness against benign usability. These findings support cost- and utility-aware evaluation as a necessary complement to success-rate-only LALM safety benchmarks.

Related papers

• Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Models [29.672999404488838]
  Existing jailbreak paradigms predominantly treat audio as a carrier for malicious payloads.<n>We propose a new paradigm in which the role of audio shifts from content injection to safety alignment interference.<n>We show that interference audio, benign in content but infused with specific ALS, can serve as a universal jailbreak trigger.
  arXiv  Detail & Related papers  (2026-05-18T10:10:31Z)
• Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injection [22.306688903148046]
  Large audio-language models (LALMs) power intelligent voice interactions by tightly integrating audio and text.<n>We propose textitAudioHijack, a framework that generates context-agnostic and imperceptible adversarial audio to hijack LALMs.<n>Experiments on 13 state-of-the-art LALMs show consistent hijacking across 6 categories misbehavior.
  arXiv  Detail & Related papers  (2026-04-16T04:22:11Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• ENJ: Optimizing Noise with Genetic Algorithms to Jailbreak LSMs [61.09812971042288]
  Evolutionary Noise Jailbreak (ENJ)<n>This paper proposes a genetic algorithm to transform environmental noise from a passive interference into an actively optimizable attack carrier for jailbreaking LSMs.<n>Experiments on multiple mainstream speech models show that ENJ's attack effectiveness is significantly superior to existing baseline methods.
  arXiv  Detail & Related papers  (2025-09-14T06:39:38Z)
• Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study [5.843063647136238]
  This study systematically evaluates five leading LALMs across four attack scenarios.<n>No single model consistently outperforms others across all attack types.<n>A negative correlation between instruction-following capability and robustness suggests models adhering strictly to instructions may be more susceptible.
  arXiv  Detail & Related papers  (2025-05-26T07:08:38Z)
• Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models [19.373533532464915]
  We introduce AJailBench, the first benchmark specifically designed to evaluate jailbreak vulnerabilities in LAMs.<n>We use this dataset to evaluate several state-of-the-art LAMs and reveal that none exhibit consistent robustness across attacks.<n>Our findings demonstrate that even small, semantically preserved perturbations can significantly reduce the safety performance of leading LAMs.
  arXiv  Detail & Related papers  (2025-05-21T11:47:47Z)
• SPIRIT: Patching Speech Language Models against Jailbreak Attacks [27.79621464198623]
  Speech Language Models (SLMs) enable natural interactions via spoken instructions.<n>We analyze adversarial attacks and find that SLMs are substantially more vulnerable to jailbreak attacks.<n>To improve security, we propose post-hoc patching defenses used to intervene during inference.
  arXiv  Detail & Related papers  (2025-05-18T21:51:24Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)
• LLM-Safety Evaluations Lack Robustness [58.334290876531036]
  We argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of noise.<n>We propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers.
  arXiv  Detail & Related papers  (2025-03-04T12:55:07Z)
• AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens [83.08119913279488]
  We present a systematic analysis of the dependency relationships in jailbreak attack and defense techniques. We propose three comprehensive, automated, and logical frameworks. We show that the proposed ensemble jailbreak attack and defense framework significantly outperforms existing research.
  arXiv  Detail & Related papers  (2024-06-06T07:24:41Z)
• Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing [107.97160023681184]
  Aligned large language models (LLMs) are vulnerable to jailbreaking attacks. We propose SEMANTICSMOOTH, a smoothing-based defense that aggregates predictions of semantically transformed copies of a given input prompt.
  arXiv  Detail & Related papers  (2024-02-25T20:36:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.