Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms

• URL: http://arxiv.org/abs/2605.05224v1
• Date: Sat, 18 Apr 2026 11:00:56 GMT
• Title: Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms
• Abstract summary: Unlearnable examples (UEs) induce models to overfit non-semantic noise, thereby weakening semantic extraction capabilities.<n>We propose a hierarchical deception strategy, Shallow Semantic Camouflage (SSC), that confines the generation process to a semantically valid subspace.
• Score: 33.54383411837643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The unauthorized use of personal data in model training has emerged as a growing privacy threat. Unlearnable examples (UEs) address this issue by embedding imperceptible perturbations into benign examples to obstruct feature learning. However, existing studies mainly evaluate UEs under from-scratch training settings, leaving their behavior under the widely adopted pretraining-finetuning (PF) paradigm largely unexplored. In this work, we provide the first systematic investigation of unlearnable examples across diverse training paradigms. Our analysis reveals that loading and freezing pretrained weights significantly weakens the effectiveness of existing UEs methods. We further explain these findings through semantic filtering: while UEs tend to induce models to overfit non-semantic noise, thereby weakening their semantic extraction capabilities, under the PF paradigm, frozen shallow layers preserve data semantics, effectively filtering out distracting information like unlearnable noise. Guided by these insights, we propose a hierarchical deception strategy, Shallow Semantic Camouflage (SSC), that confines the generation process to a semantically valid subspace, aiming to bypass the semantic suppression introduced by pretrained weights. Extensive experiments demonstrate that our method consistently preserves data unlearnability even under challenging training paradigms, such as shallow-layer freezing and semantic-focused pretraining (SF-Pretrain), bridging the critical gap in pretrain-based unlearnable learning.

Related papers

• UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation [90.0896070825457]
  We propose a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes.<n>Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content.<n>We develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead.
  arXiv  Detail & Related papers  (2026-07-18T14:03:39Z)
• When Priors Backfire: On the Vulnerability of Unlearnable Examples to Pretraining [28.704214093822472]
  Unlearnable Examples (UEs) serve as a data protection strategy that generates imperceptible perturbations to mislead models into learning spurious correlations instead of underlying semantics.<n>In this paper, we uncover a fundamental vulnerability of UEs that emerges when learning starts from a pretrained model.<n>We propose BAIT (Binding Artificial perturbations to Incorrect Targets), a novel bi-level optimization formulation.
  arXiv  Detail & Related papers  (2026-03-05T02:10:40Z)
• Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise [51.179816451161635]
  Diffusion models have achieved remarkable progress in image generation, but their increasing deployment raises serious concerns about privacy.<n>In this work, we utilize a critical yet overlooked vulnerability: the widely used noise schedules fail to fully eliminate semantic information in the images.<n>We propose a simple yet effective membership inference attack, which injects semantic information into the initial noise and infers membership by analyzing the model's generation result.
  arXiv  Detail & Related papers  (2026-01-29T12:29:01Z)
• Impact of Noisy Supervision in Foundation Model Learning [91.56591923244943]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.<n>We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Fast Adversarial Training against Textual Adversarial Attacks [11.023035222098008]
  We propose a Fast Adversarial Training (FAT) method to improve the model robustness in the synonym-unaware scenario. FAT uses single-step and multi-step gradient ascent to craft adversarial examples in the embedding space. Experiments demonstrate that FAT significantly boosts the robustness of BERT models in the synonym-unaware scenario.
  arXiv  Detail & Related papers  (2024-01-23T03:03:57Z)
• Segue: Side-information Guided Generative Unlearnable Examples for Facial Privacy Protection in Real World [64.4289385463226]
  We propose Segue: Side-information guided generative unlearnable examples. To improve transferability, we introduce side information such as true labels and pseudo labels. It can resist JPEG compression, adversarial training, and some standard data augmentations.
  arXiv  Detail & Related papers  (2023-10-24T06:22:37Z)
• Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks [91.15120211190519]
  This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks. We propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise.
  arXiv  Detail & Related papers  (2023-09-29T06:18:15Z)
• Unlearnable Examples Give a False Sense of Data Privacy: Understanding and Relearning [31.2971146235291]
  Unlearnable examples generate unlearnable examples by adding imperceptible perturbations to public publishing data.<n>We propose Progressive Staged Training, a self-adaptive training framework specially designed to break unlearnable examples.<n>Our method circumvents the unlearnability of all state-of-the-art methods in the literature.
  arXiv  Detail & Related papers  (2023-06-03T09:36:16Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling [27.602336774468]
  Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. We propose a semantic awareness structure transferring method for training perturbation-robust slot filling models.
  arXiv  Detail & Related papers  (2022-08-24T13:01:00Z)
• Semantics-Preserving Adversarial Training [12.242659601882147]
  Adversarial training is a technique that improves adversarial robustness of a deep neural network (DNN) by including adversarial examples in the training data. We propose semantics-preserving adversarial training (SPAT) which encourages perturbation on the pixels that are shared among all classes. Experiment results show that SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2020-09-23T07:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.