One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries

• URL: http://arxiv.org/abs/2605.14605v1
• Date: Thu, 14 May 2026 09:22:14 GMT
• Title: One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries
• Abstract summary: Model providers increasingly release open weights or allow users to fine-tune foundation models through APIs.<n>Recent defenses aim to make models robust to such malicious fine-tuning, but they are largely evaluated only against fixed attacks that do not account for the defense.<n>We show that these robustness claims are incomplete.<n> Surveying 15 recent defenses, we identify several defense mechanisms and show that they share a single weakness: they obscure or misdirect the path to harmful behavior without removing the behavior itself.<n>We then develop a unified adaptive attack that breaks defenses across all defense mechanisms.
• Score: 2.657356138730743
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Model providers increasingly release open weights or allow users to fine-tune foundation models through APIs. Although these models are safety-aligned before release, their safeguards can often be removed by fine-tuning on harmful data. Recent defenses aim to make models robust to such malicious fine-tuning, but they are largely evaluated only against fixed attacks that do not account for the defense. We show that these robustness claims are incomplete. Surveying 15 recent defenses, we identify several defense mechanisms and show that they share a single weakness: they obscure or misdirect the path to harmful behavior without removing the behavior itself. We then develop a unified adaptive attack that breaks defenses across all defense mechanisms. Our results show that current approaches do not provide robust security; they mainly stop the attacks they were designed against. We hope that our unified adaptive adversary for this domain will help future researchers and practitioners stress-test new defenses before deployment.

Related papers

• Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks [21.29943620687951]
  We show that open-weight safeguards are susceptible to simpler strategies that have not been systematically evaluated against these safeguards.<n>We introduce abliteration-resistant tuning (ART), which incorporates an abliteration-based objective into training.
  arXiv  Detail & Related papers  (2026-05-26T04:18:42Z)
• The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections [74.60337113759313]
  Current defenses against jailbreaks and prompt injections are typically evaluated against a static set of harmful attack strings.<n>We argue that this evaluation process is flawed. Instead, we should evaluate defenses against adaptive attackers who explicitly modify their attack strategy to counter a defense's design.
  arXiv  Detail & Related papers  (2025-10-10T05:51:04Z)
• Boosting Active Defense Persistence: A Two-Stage Defense Framework Combining Interruption and Poisoning Against Deepfake [14.10448497174767]
  We argue that an effective defense not only distorts forged content but also blocks the model's ability to adapt.<n>To achieve this, we propose an innovative Two-Stage Defense Framework (TSDF)<n>Our framework shows a strong dual defense capability, which can improve the persistence of active defense.
  arXiv  Detail & Related papers  (2025-08-11T09:26:48Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• Fundamental Limitations in Pointwise Defences of LLM Finetuning APIs [56.84338097581665]
  We show that defences of fine-tuning APIs are fundamentally limited in their ability to prevent fine-tuning attacks.<n>We construct 'pointwise-undetectable' attacks that repurpose entropy in benign model outputs to covertly transmit dangerous knowledge.<n>We test our attacks against the OpenAI fine-tuning API, finding they succeed in eliciting answers to harmful multiple-choice questions.
  arXiv  Detail & Related papers  (2025-02-20T18:45:01Z)
• Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity [46.83576558654259]
  We argue that current safety fine-tuning is very similar to a traditional cat-and-mouse game between attackers and defenders in cybersecurity.<n>We show how current defenses are insufficient to prevent new adversarial jailbreak attacks, reward hacking, and loss of control problems.
  arXiv  Detail & Related papers  (2025-01-19T21:49:42Z)
• Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators [49.52538232104449]
  It is becoming increasingly imperative to design robust ML defenses. Recent work has found that many defenses that initially resist state-of-the-art attacks can be broken by an adaptive adversary. We take steps to simplify the design of defenses and argue that white-box defenses should eschew randomness when possible.
  arXiv  Detail & Related papers  (2023-02-27T01:33:31Z)
• Evaluating the Adversarial Robustness of Adaptive Test-time Defenses [60.55448652445904]
  We categorize such adaptive testtime defenses and explain their potential benefits and drawbacks. Unfortunately, none significantly improve upon static models when evaluated appropriately. Some even weaken the underlying static model while simultaneously increasing inference cost.
  arXiv  Detail & Related papers  (2022-02-28T12:11:40Z)
• What Doesn't Kill You Makes You Robust(er): Adversarial Training against Poisons and Backdoors [57.040948169155925]
  We extend the adversarial training framework to defend against (training-time) poisoning and backdoor attacks. Our method desensitizes networks to the effects of poisoning by creating poisons during training and injecting them into training batches. We show that this defense withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses.
  arXiv  Detail & Related papers  (2021-02-26T17:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.