Trusted Weights, Treacherous Optimizations? Optimization-Triggered Backdoor Attacks on LLMs

• URL: http://arxiv.org/abs/2605.20641v1
• Date: Wed, 20 May 2026 02:55:56 GMT
• Title: Trusted Weights, Treacherous Optimizations? Optimization-Triggered Backdoor Attacks on LLMs
• Abstract summary: We propose a unified optimization-triggered attack framework comprising two complementary strategies.<n>One strategy flips predictions for specific inputs only when the model is compiled.<n>The other uses a universal trigger that remains dormant under uncompiled execution but hijacks arbitrary inputs once compilation optimization is applied.
• Score: 23.226094056337075
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inference optimization is a vital technique for deploying LLMs at scale. Compilation is the most widely adopted optimization technique for LLMs. While it assumes semantic equivalence between the original and compiled graphs, we first uncover its numerical side effects can be maliciously exploited to implant stealthy backdoors in LLMs. We propose a unified optimization-triggered attack framework comprising two complementary strategies. Without any modification to the compiler or hardware, one strategy flips predictions for specific inputs only when the model is compiled, while the other uses a universal trigger that remains dormant under uncompiled execution but hijacks arbitrary inputs once compilation optimization is applied. Both attacks bypass standard safety evaluations run without compilation. We empirically demonstrate that these optimization-triggered backdoors achieve attack success rates averaging 90% across four mainstream open-source LLMs and four tasks, while clean accuracy is preserved at nearly 100% under all settings. Our findings reveal a novel attack surface at the intersection of optimization and security in the LLM deployment pipeline, and we investigate practical defenses to mitigate this threat.

Related papers

• Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making [2.4336422314992916]
  Large language models (LLMs) are increasingly used in cybersecurity, yet it remains unclear whether they can perform structured security reasoning.<n>We study this question in the context of defence selection over attack graphs derived from real-world threat scenarios.<n>Our findings show that LLMs can approximate structured cybersecurity reasoning under controlled representations, but do not apply it robustly.
  arXiv  Detail & Related papers  (2026-08-21T10:39:42Z)
• RogueMerge: Robust and Unified Attacks against LLM Model Merging [16.43903795600829]
  RogueMerge is a principled, unified framework for model merging attacks.<n>It consistently outperforms existing attacks across four threats.<n>It remains stable across diverse merging settings and resists standard defenses.
  arXiv  Detail & Related papers  (2026-06-02T08:54:37Z)
• Adversarial Reframing: A Framework for Targeted Generation in Language Models [4.231181719953953]
  Large Language Models (LLMs) are widely deployed in diverse real-world settings, yet remain vulnerable to jailbreaking, where prompt-based attacks safety filters.<n>We present THREAT (edversa bypass generation via Reframing) as a tool for proactively strengthening the safety of foundation models.
  arXiv  Detail & Related papers  (2026-05-20T19:31:07Z)
• TASO: Jailbreak LLMs via Alternative Template and Suffix Optimization [52.01940078632388]
  We introduce TASO, a novel jailbreak method that optimize both a template and a suffix in an alternating manner.<n>We evaluate the effectiveness of TASO on benchmark datasets on 24 leading LLMs.
  arXiv  Detail & Related papers  (2025-11-23T18:49:27Z)
• PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization [0.0]
  This paper introduces a novel framework for hardening system prompts through shield appending.<n>We leverage an LLM-as-optimizer to search the space of possible SHIELDs, seeking to minimize a leakage metric derived from a suite of adversarial attacks.<n>We demonstrate empirically that our optimized SHIELDs significantly reduce prompt leakage against a comprehensive set of extraction attacks.
  arXiv  Detail & Related papers  (2025-11-20T10:25:45Z)
• ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models [55.93380086403591]
  Generative large language models are vulnerable to backdoor attacks.<n>$textitELBA-Bench$ allows attackers to inject backdoor through parameter efficient fine-tuning.<n>$textitELBA-Bench$ provides over 1300 experiments.
  arXiv  Detail & Related papers  (2025-02-22T12:55:28Z)
• An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizer [33.67942887761857]
  We present ECLIPSE, a novel and efficient black-box jailbreaking method utilizing optimizable suffixes.<n>We employ task prompts to translate jailbreaking goals into natural language instructions, which guides the LLM to generate adversarial suffixes for malicious queries.<n>ECLIPSE achieves an average attack success rate (ASR) of 0.92 across three open-source LLMs and GPT-3.5-Turbo, significantly surpassing GCG in 2.4 times.
  arXiv  Detail & Related papers  (2024-08-21T03:35:24Z)
• ShadowCode: Towards (Automatic) External Prompt Injection Attack against Code LLMs [56.46702494338318]
  This paper introduces a new attack paradigm: (automatic) external prompt injection against code-oriented large language models.<n>We propose ShadowCode, a simple yet effective method that automatically generates induced perturbations based on code simulation.<n>We evaluate our method across 13 distinct malicious objectives, generating 31 threat cases spanning three popular programming languages.
  arXiv  Detail & Related papers  (2024-07-12T10:59:32Z)
• Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA [2.4578723416255754]
  We introduce QROA, a black-box jailbreak method to identify adversarial suffixes when appended to a malicious instruction.<n>Unlike existing suffix-based jailbreak approaches, QROA does not require access to the model's logit or any other internal information.<n>We also propose QROA-UNV, an extension that identifies universal adversarial suffixes for individual models.
  arXiv  Detail & Related papers  (2024-06-04T07:27:36Z)
• PAL: Proxy-Guided Black-Box Attack on Large Language Models [55.57987172146731]
  Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated capabilities to generate harmful content when manipulated. We introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art.
  arXiv  Detail & Related papers  (2024-02-15T02:54:49Z)
• Instruction Backdoor Attacks Against Customized LLMs [37.92008159382539]
  We propose the first instruction backdoor attacks against applications integrated with untrusted customized LLMs. Our attack includes 3 levels of attacks: word-level, syntax-level, and semantic-level, which adopt different types of triggers with progressive stealthiness. We propose two defense strategies and demonstrate their effectiveness in reducing such attacks.
  arXiv  Detail & Related papers  (2024-02-14T13:47:35Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)
• Versatile Weight Attack via Flipping Limited Bits [68.45224286690932]
  We study a novel attack paradigm, which modifies model parameters in the deployment stage. Considering the effectiveness and stealthiness goals, we provide a general formulation to perform the bit-flip based weight attack. We present two cases of the general formulation with different malicious purposes, i.e., single sample attack (SSA) and triggered samples attack (TSA)
  arXiv  Detail & Related papers  (2022-07-25T03:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.