DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation

• URL: http://arxiv.org/abs/2603.07835v1
• Date: Sun, 08 Mar 2026 22:38:30 GMT
• Title: DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation
• Abstract summary: We introduce a taxonomy of three defense categories -- output perturbation, data poisoning, and information throttling.<n>We evaluate nine defense configurations using a standardized pipeline with Qwen3-14B as teacher and Qwen2.5-7B-Instruct as student.<n>Our results reveal that, in a same-family distillation setting against a naive attacker, most output-level defenses are surprisingly ineffective.
• Score: 5.647839536820347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge distillation from proprietary LLM APIs poses a growing threat to model providers, yet defenses against this attack remain fragmented and unevaluated. We present DistillGuard, a framework for systematically evaluating output-level defenses against LLM knowledge distillation. We introduce a taxonomy of three defense categories -- output perturbation, data poisoning, and information throttling -- and evaluate nine defense configurations using a standardized pipeline with Qwen3-14B as teacher and Qwen2.5-7B-Instruct as student across three benchmarks (MATH-500, HumanEval+, MT-Bench). Our results reveal that, in a same-family distillation setting against a naive attacker, most output-level defenses are surprisingly ineffective: paraphrasing-based perturbation barely degrades distilled student quality, and data poisoning primarily impairs conversational fluency while leaving task-specific capabilities intact. Only chain-of-thought removal substantially impairs mathematical reasoning (31.4\% vs.\ 67.8\% baseline), though code generation remains unaffected. These findings demonstrate that the effectiveness of distillation defenses is highly task-dependent and that current output-level approaches are insufficient to broadly prevent knowledge theft.

Related papers

• What Does It Mean to Break a Distillation Defense? [33.607700445538136]
  Black-box LLMs are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs.<n>We propose a threat model framework that describes attackers along three dimensions: a query budget, a data budget, and an interface profile.<n>We argue that future work on distillation defenses, along with any governance or policy frameworks built around them, should explicitly specify and stress-test attacker capabilities.
  arXiv  Detail & Related papers  (2026-06-23T18:13:02Z)
• Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility [13.916452483499185]
  Anti-distillation (AD) has been proposed to generate defensive outputs that hinder distillation effectiveness.<n>We propose textbfunderlineSkeleton-textbfunderlineGuided textbfunderlineReasoning textbfunderlineEditing (SGRE)<n>In the answer stage, the teacher model first generates clean reasoning traces, preserving the original reasoning accuracy.<n>In the editing stage, we introduce a three-stage strategy consisting of reasoning skeleton extraction, skeleton graph coarsening, and skeleton verbal
  arXiv  Detail & Related papers  (2026-05-12T04:36:20Z)
• Dummy-Aware Weighted Attack (DAWA): Breaking the Safe Sink in Dummy Class Defenses [56.390980653401506]
  Dummy Classes-based defenses achieve significantly overestimated robustness under conventional evaluation strategies like AutoAttack.<n>We propose Dummy-Aware Weighted Attack (DAWA), a novel evaluation method that simultaneously targets both the true label and dummy label.<n>Our work provides a more reliable benchmark for evaluating this emerging class of defenses and highlights the need for continuous evolution of robustness assessment methodologies.
  arXiv  Detail & Related papers  (2026-03-31T02:49:17Z)
• Agentic Knowledge Distillation: Autonomous Training of Small Language Models for SMS Threat Detection [0.4899818550820574]
  Agentic Knowledge Distillation consists of a powerful LLM that fine-tunes a smaller student SLM, deployable for security tasks without human intervention.<n>We compare four LLMs in this teacher role on SMS spam/smishing detection with two student SLMs.<n>Our results show that performance varies substantially depending on the teacher LLM, with the best configuration achieving 94.31% accuracy and 96.25% recall.
  arXiv  Detail & Related papers  (2026-02-11T13:57:56Z)
• Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective [52.25797439810419]
  Existing defenses focus exclusively on text-based distillation, leaving the important logit-based distillation largely unexplored.<n>We characterize distillation-relevant information in teacher outputs using the conditional mutual information (CMI) between teacher logits and input queries conditioned on ground-truth labels.<n>We derive a CMI-inspired anti-distillation objective to optimize this transformation, which effectively removes distillation-relevant information while preserving output utility.
  arXiv  Detail & Related papers  (2026-02-03T11:16:59Z)
• SecureLearn - An Attack-agnostic Defense for Multiclass Machine Learning Against Data Poisoning Attacks [0.0]
  Existing defenses are mostly designed to mitigate specific poisoning attacks or are aligned with particular ML algorithms.<n>This paper proposes SecureLearn, a two-layer attack-agnostic defense to defend multiclass models from poisoning attacks.<n>For neural networks, SecureLearn achieved 97% recall and F1-score against all selected poisoning attacks.
  arXiv  Detail & Related papers  (2025-10-25T12:35:45Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation [49.58082402742583]
  Large Language Models (LLMs) represent substantial intellectual and economic investments.<n>LLMs can inadvertently facilitate model imitation via knowledge distillation (KD)<n>This paper introduces an effective and efficient Defensive Output Generation (DOGe) strategy.
  arXiv  Detail & Related papers  (2025-05-26T04:31:38Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense [56.32083100401117]
  The vulnerability of Vision Large Language Models (VLLMs) to jailbreak attacks appears as no surprise.<n>Recent defense mechanisms against these attacks have reached near-saturation performance on benchmark evaluations.
  arXiv  Detail & Related papers  (2024-11-13T07:57:19Z)
• Turning Generative Models Degenerate: The Power of Data Poisoning Attacks [10.36389246679405]
  Malicious actors can introduce backdoors through poisoning attacks to generate undesirable outputs. We conduct an investigation of various poisoning techniques targeting the large language models' fine-tuning phase via the Efficient Fine-Tuning (PEFT) method. Our study presents the first systematic approach to understanding poisoning attacks targeting NLG tasks during fine-tuning via PEFT.
  arXiv  Detail & Related papers  (2024-07-17T03:02:15Z)
• Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation [122.83280749890078]
  We propose an improved certified defense against general poisoning attacks, namely Finite Aggregation. In contrast to DPA, which directly splits the training set into disjoint subsets, our method first splits the training set into smaller disjoint subsets. We offer an alternative view of our method, bridging the designs of deterministic and aggregation-based certified defenses.
  arXiv  Detail & Related papers  (2022-02-05T20:08:58Z)
• How Robust are Randomized Smoothing based Defenses to Data Poisoning? [66.80663779176979]
  We present a previously unrecognized threat to robust machine learning models that highlights the importance of training-data quality. We propose a novel bilevel optimization-based data poisoning attack that degrades the robustness guarantees of certifiably robust classifiers. Our attack is effective even when the victim trains the models from scratch using state-of-the-art robust training methods.
  arXiv  Detail & Related papers  (2020-12-02T15:30:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.