MechAudit-40: White-Box Auditing across 40 LLM Attack Mechanisms

• URL: http://arxiv.org/abs/2609.06612v1
• Date: Sun, 06 Sep 2026 13:57:41 GMT
• Title: MechAudit-40: White-Box Auditing across 40 LLM Attack Mechanisms
• Abstract summary: MechAudit-40 is a systematic evaluation of 40 attack mechanisms across five open-weight model architectures.<n>Attacks induce structured multi-depth trajectories rather than isolated layer spikes.<n> MechAudit detects 81.1% of held-out attack executions at a 0.70% false-positive rate and maintains 78.1% recall when an entire functional category is withheld.
• Score: 16.312485460260056
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While LLM attacks span prompt optimization, multi-turn context manipulation, retrieval poisoning, and model backdoors, white-box defenses are typically evaluated on isolated attack families. Consequently, whether heterogeneous attacks leave internal representation shifts that generalize to unseen threat mechanisms remains unknown. We present MechAudit-40, a systematic evaluation of 40 attack mechanisms across five open-weight model architectures. Threat-specific success criteria, 100,000 matched clean-attack representation pairs, predefined categories, and grouped holdouts isolate genuine attack-induced displacement from target scale, corpus bias, and data-leakage shortcuts. Across this testbed, attacks induce structured multi-depth trajectories rather than isolated layer spikes. While raw peaks are non-portable across architectures, target-calibrated profiles preserve transferable geometric signatures: under complete mechanism holdout, hidden states alone recover the threat category of unseen attacks with 82.5% accuracy. Guided by this finding, we design MechAudit, a runtime auditor that operates under strict zero-oracle constraints without requiring clean baseline traces or attack metadata. MechAudit detects 81.1% of held-out attack executions at a 0.70% false-positive rate and maintains 78.1% recall when an entire functional category is withheld. In matched comparisons, MechAudit is the only detector that avoids mechanism-level coverage collapse, maintaining over 50% recall across all 40 mechanisms. Internal representations thus support cross-mechanism attack-exposure auditing against calibrated benign references, but decouple from downstream task compromise and parameter integrity.

Related papers

• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy [51.758416625168]
  Randomized smoothing is a powerful tool for certifying robustness to adversarial perturbations.<n>We propose a framework for certification of complex, composed mechanisms.<n>We provide a principled and general framework for using composite mechanisms to certify robustness under complex threat models.
  arXiv  Detail & Related papers  (2026-05-20T22:17:29Z)
• Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks [0.0]
  We introduce a reusable framework for auditing whether attack benchmarks collectively cover the threat surface.<n>Applying it to six public benchmarks reveals that the three primary frameworks occupy non-overlapping cells covering at most 25% of the matrix.<n>The corpus of 2,521 unique attack groups reveals pervasive naming fragmentation and heavy concentration in Safety & Alignment Bypass.
  arXiv  Detail & Related papers  (2026-05-14T17:30:36Z)
• Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization [51.11398557420066]
  We introduce Metis, a framework that reformulates jailbreaking as inference-time policy optimization.<n>Metis achieves the strongest average Attack Success Rate (ASR) among compared methods at 89.2%.<n>By replacing redundant exploration with directed optimization, Metis reduces token costs by an average of 8.2x and up to 11.4x.
  arXiv  Detail & Related papers  (2026-05-11T06:45:00Z)
• AI Evasion and Impersonation Attacks on Facial Re-Identification with Activation Map Explanations [13.564481603680838]
  This paper introduces a novel framework for generating adversarial patches capable of both evasion and impersonation attacks.<n>We use a conditional encoder-decoder network to synthesize adversarial patches in a single forward pass, guided by multi-scale features from source and target images.<n>Our framework achieves a success rate of 27% on CelebA-HQ, competing with other patch-based methods.
  arXiv  Detail & Related papers  (2026-03-16T15:12:41Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering [0.0]
  We present TRYLOCK, the first defense-in-depth architecture that combines four mechanisms across the inference stack.<n>On Mistral-7B-Instruct evaluated against a 249-prompt attack set, TRYLOCK achieves 88.0% relative ASR reduction.
  arXiv  Detail & Related papers  (2026-01-06T03:02:20Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• Replicating TEMPEST at Scale: Multi-Turn Adversarial Attacks Against Trillion-Parameter Frontier Models [0.0]
  This study employed the TEMPEST multi-turn attack framework to evaluate ten frontier models from eight vendors across 1,000 harmful behaviors.<n>Six models achieved 96% to 100% attack success rate (ASR), while four showed meaningful resistance, with ASR ranging from 42% to 78%.
  arXiv  Detail & Related papers  (2025-12-08T00:30:40Z)
• ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models [55.93380086403591]
  Generative large language models are vulnerable to backdoor attacks.<n>$textitELBA-Bench$ allows attackers to inject backdoor through parameter efficient fine-tuning.<n>$textitELBA-Bench$ provides over 1300 experiments.
  arXiv  Detail & Related papers  (2025-02-22T12:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.