Protecting the Trace: A Principled Black-Box Approach Against Distillation Attacks

• URL: http://arxiv.org/abs/2604.23238v1
• Date: Sat, 25 Apr 2026 10:22:09 GMT
• Title: Protecting the Trace: A Principled Black-Box Approach Against Distillation Attacks
• Abstract summary: Antidistillation methods aim to poison reasoning traces to hinder downstream student model learning while maintaining teacher performance.<n>Current techniques lack theoretical grounding, requiring either heavy fine-tuning or access to student model proxies for gradient based attacks.<n>We propose ttexttTraceGuard, an efficient, post-generation black-box method to poison sentences with high importance for teacher reasoning.
• Score: 12.75752756558549
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Frontier models push the boundaries of what is learnable at extreme computational costs, yet distillation via sampling reasoning traces exposes closed-source frontier models to adversarial third parties who can bypass their guardrails and misappropriate their capabilities, raising safety, security, and intellectual privacy concerns. To address this, there is growing interest in building antidistillation methods, which aim to poison reasoning traces to hinder downstream student model learning while maintaining teacher performance. However, current techniques lack theoretical grounding, requiring either heavy fine-tuning or access to student model proxies for gradient based attacks, and often lead to a significant teacher performance degradation. In this work, we present a theoretical formulation of antidistillation as a Stackelberg game, grounding a problem that has so far largely been approached heuristically. Guided by the desired design properties our formulation reveals, we propose \texttt{TraceGuard}, an efficient, post-generation black-box method to poison sentences with high importance for teacher reasoning. Our work offers a scalable solution to share model insights safely, ensuring that the advancement of reasoning capabilities does not come at the cost of intellectual privacy or AI safety alignment.

Related papers

• Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility [13.916452483499185]
  Anti-distillation (AD) has been proposed to generate defensive outputs that hinder distillation effectiveness.<n>We propose textbfunderlineSkeleton-textbfunderlineGuided textbfunderlineReasoning textbfunderlineEditing (SGRE)<n>In the answer stage, the teacher model first generates clean reasoning traces, preserving the original reasoning accuracy.<n>In the editing stage, we introduce a three-stage strategy consisting of reasoning skeleton extraction, skeleton graph coarsening, and skeleton verbal
  arXiv  Detail & Related papers  (2026-05-12T04:36:20Z)
• Protecting Language Models Against Unauthorized Distillation through Trace Rewriting [31.05181251141126]
  Authoritarian use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models.<n>We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence.<n>Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance.
  arXiv  Detail & Related papers  (2026-02-16T19:40:07Z)
• Beyond Memorization: Gradient Projection Enables Selective Learning in Diffusion Models [3.4064487905075294]
  Memorization in large-scale text-to-image diffusion models poses significant security and intellectual property risks.<n>We introduce a Gradient Projection Framework designed to enforce a stringent requirement of concept-level feature exclusion.<n>Our approach establishes a new paradigm for IP-safe and privacy-preserving generative AI.
  arXiv  Detail & Related papers  (2025-12-12T00:50:38Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• Exploiting Edge Features for Transferable Adversarial Attacks in Distributed Machine Learning [54.26807397329468]
  This work explores a previously overlooked vulnerability in distributed deep learning systems.<n>An adversary who intercepts the intermediate features transmitted between them can still pose a serious threat.<n>We propose an exploitation strategy specifically designed for distributed settings.
  arXiv  Detail & Related papers  (2025-07-09T20:09:00Z)
• How to Backdoor the Knowledge Distillation [10.478504819079548]
  We introduce a novel attack methodology that strategically poisons the distillation dataset with adversarial examples embedded with backdoor triggers.<n>This technique allows for the stealthy compromise of the student model while maintaining the integrity of the teacher model.<n>Our findings reveal previously unrecognized vulnerabilities and pave the way for future research aimed at securing knowledge distillation processes.
  arXiv  Detail & Related papers  (2025-04-30T05:19:23Z)
• Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions [50.40122190627256]
  We introduce POATE, a novel jailbreak technique that harnesses contrastive reasoning to provoke unethical responses.<n>PoATE crafts semantically opposing intents and integrates them with adversarial templates, steering models toward harmful outputs with remarkable subtlety.<n>To counter this, we propose Intent-Aware CoT and Reverse Thinking CoT, which decompose queries to detect malicious intent and reason in reverse to evaluate and reject harmful responses.
  arXiv  Detail & Related papers  (2025-01-03T15:40:03Z)
• Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examples [1.1820990818670631]
  This work is the first to provide provable guarantees on the success of knowledge distillation-based attack on classification neural networks. We prove that if the student model has enough learning capabilities, the attack on the teacher model is guaranteed to be found within the finite number of distillation iterations.
  arXiv  Detail & Related papers  (2024-10-21T11:06:56Z)
• Adversarial Sparse Teacher: Defense Against Distillation-Based Model Stealing Attacks Using Adversarial Examples [2.0257616108612373]
  Adversarial Sparse Teacher (AST) is a robust defense method against distillation-based model stealing attacks. Our approach trains a teacher model using adversarial examples to produce sparse logit responses and increase the entropy of the output distribution.
  arXiv  Detail & Related papers  (2024-03-08T09:43:27Z)
• Performance-lossless Black-box Model Watermarking [69.22653003059031]
  We propose a branch backdoor-based model watermarking protocol to protect model intellectual property. In addition, we analyze the potential threats to the protocol and provide a secure and feasible watermarking instance for language models.
  arXiv  Detail & Related papers  (2023-12-11T16:14:04Z)
• Isolation and Induction: Training Robust Deep Neural Networks against Model Stealing Attacks [51.51023951695014]
  Existing model stealing defenses add deceptive perturbations to the victim's posterior probabilities to mislead the attackers. This paper proposes Isolation and Induction (InI), a novel and effective training framework for model stealing defenses. In contrast to adding perturbations over model predictions that harm the benign accuracy, we train models to produce uninformative outputs against stealing queries.
  arXiv  Detail & Related papers  (2023-08-02T05:54:01Z)
• Deep Model Intellectual Property Protection via Deep Watermarking [122.87871873450014]
  Deep neural networks are exposed to serious IP infringement risks. Given a target deep model, if the attacker knows its full information, it can be easily stolen by fine-tuning. We propose a new model watermarking framework for protecting deep networks trained for low-level computer vision or image processing tasks.
  arXiv  Detail & Related papers  (2021-03-08T18:58:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.