Defending Against Harmful Supervision Hidden in Benign Samples

• URL: http://arxiv.org/abs/2606.30263v1
• Date: Mon, 29 Jun 2026 13:11:49 GMT
• Title: Defending Against Harmful Supervision Hidden in Benign Samples
• Abstract summary: Existing defenses are effective when harmful content is explicitly mixed into downstream fine-tuning data, but crafted samples can instead hide harmful supervision inside benign tasks.<n>We propose Embedded Attack, where harmful QA pairs are embedded within benign training samples, and show that representative guardrails often fail to detect them at the example level.
• Score: 61.51852972402142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing defenses are effective when harmful content is explicitly mixed into downstream fine-tuning data, but crafted samples can instead hide harmful supervision inside benign tasks. We propose Embedded Attack, where harmful QA pairs are embedded within benign training samples, and show that representative guardrails often fail to detect them at the example level. To address this, we propose Dual-Reference SFT (DR-SFT), which adapts DPO-style contrastive objective design to SFT through token-level regularization, mitigating harmful fine-tuning beyond coarse data filtering.

Related papers

• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• Dashed Line Defense: Plug-And-Play Defense Against Adaptive Score-Based Query Attacks [3.206339985805037]
  Dashed Line Defense (DLD) is a plug-and-play post-processing method specifically designed to withstand adaptive query strategies.<n>By introducing ambiguity in how the observed loss reflects the true adversarial strength of candidate examples, DLD prevents attackers from reliably analyzing and adapting their queries.<n>We provide theoretical guarantees of DLD's defense capability and validate its effectiveness through experiments on ImageNet.
  arXiv  Detail & Related papers  (2026-02-09T14:02:32Z)
• Defenses Against Prompt Attacks Learn Surface Heuristics [40.392588465939106]
  Large language models (LLMs) are increasingly deployed in security-sensitive applications.<n>LLMs may override intended logic when adversarial instructions appear in user queries or retrieved content.<n>Recent defenses rely on supervised fine-tuning with benign and malicious labels.
  arXiv  Detail & Related papers  (2026-01-12T04:12:48Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• A Few Large Shifts: Layer-Inconsistency Based Minimal Overhead Adversarial Example Detection [13.109309606764754]
  We introduce a plug-in detection framework that leverages internal layer-wise inconsistencies within the target model itself.<n>Our method achieves state-of-the-art detection performance with negligible computational overhead.
  arXiv  Detail & Related papers  (2025-05-19T00:48:53Z)
• STBA: Towards Evaluating the Robustness of DNNs for Query-Limited Black-box Scenario [50.37501379058119]
  We propose the Spatial Transform Black-box Attack (STBA) to craft formidable adversarial examples in the query-limited scenario. We show that STBA could effectively improve the imperceptibility of the adversarial examples and remarkably boost the attack success rate under query-limited settings.
  arXiv  Detail & Related papers  (2024-03-30T13:28:53Z)
• Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning [80.21709045433096]
  A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample. We use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
  arXiv  Detail & Related papers  (2022-11-04T13:54:02Z)
• Generating Out of Distribution Adversarial Attack using Latent Space Poisoning [5.1314136039587925]
  We propose a novel mechanism of generating adversarial examples where the actual image is not corrupted. latent space representation is utilized to tamper with the inherent structure of the image. As opposed to gradient-based attacks, the latent space poisoning exploits the inclination of classifiers to model the independent and identical distribution of the training dataset.
  arXiv  Detail & Related papers  (2020-12-09T13:05:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.