Theory of Continual Learning Against Data Poisoning Attacks

• URL: http://arxiv.org/abs/2606.29841v1
• Date: Mon, 29 Jun 2026 06:28:49 GMT
• Title: Theory of Continual Learning Against Data Poisoning Attacks
• Abstract summary: We develop a theoretical framework to analyze strategic attacks and defenses in regularization-based CL.<n>We propose a task-to-task verification mechanism to detect data poisoning and reduce cumulative bias for learning convergence.
• Score: 24.02205412427037
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning (CL), where a model is trained on a sequence of data tasks, is increasingly being adopted across key fields such as large language models and image recognition, yet it remains highly vulnerable to data poisoning that triggers learning divergence or severe excess risk. Despite these threats, a principled theoretical foundation in CL for understanding attack and defense remains lacking. In this paper, we develop a theoretical framework to analyze strategic attacks and defenses in regularization-based CL, a cornerstone of recent CL theory. By framing the adversary-defender interaction as an online zero-sum game, we first establish a fundamental performance limit: no defense succeeds when an adversary poisons a linear proportion of tasks by injecting unbounded noise or pattern shifts in regularization-based CL. We then analyze two possibly defensible scenarios: infrequent attacks and bounded noise per attack. For the former regime, we propose a task-to-task verification mechanism to detect data poisoning and reduce cumulative bias for learning convergence. For the latter regime, we derive a robust defense that minimizes the model's sensitivity to poisoned features, provably accelerating the convergence rate. Extensive experiments on realistic tasks further validate our theoretical results.

Related papers

• CSC: Turning the Adversary's Poison against Itself [23.878184947897765]
  Poisoning-based backdoor attacks pose significant threats to deep neural networks.<n>This paper conducts a comprehensive analysis of backdoor attack dynamics during model training.<n>We propose Cluster Segregation Concealment (CSC), a novel poison suppression defense.
  arXiv  Detail & Related papers  (2026-04-23T08:30:53Z)
• The Eminence in Shadow: Exploiting Feature Boundary Ambiguity for Robust Backdoor Attacks [51.468144272905135]
  Deep neural networks (DNNs) underpin critical applications yet remain vulnerable to backdoor attacks.<n>We provide a theoretical analysis targeting backdoor attacks, focusing on how sparse decision boundaries enable disproportionate model manipulation.<n>We propose Eminence, an explainable and robust black-box backdoor framework with provable theoretical guarantees and inherent stealth properties.
  arXiv  Detail & Related papers  (2025-12-11T08:09:07Z)
• Mellivora Capensis: A Backdoor-Free Training Framework on the Poisoned Dataset without Auxiliary Data [39.07360350023601]
  This paper addresses the challenges of backdoor attack countermeasures in real-world scenarios.<n>We propose a robust and clean-data-free backdoor defense framework, namely Mellivora Capensis (textttMeCa), which enables the model trainer to train a clean model on the poisoned dataset.
  arXiv  Detail & Related papers  (2024-05-21T12:20:19Z)
• FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning [98.43475653490219]
  Federated learning (FL) is susceptible to poisoning attacks. FreqFed is a novel aggregation mechanism that transforms the model updates into the frequency domain. We demonstrate that FreqFed can mitigate poisoning attacks effectively with a negligible impact on the utility of the aggregated model.
  arXiv  Detail & Related papers  (2023-12-07T16:56:24Z)
• On Practical Aspects of Aggregation Defenses against Data Poisoning Attacks [58.718697580177356]
  Attacks on deep learning models with malicious training samples are known as data poisoning. Recent advances in defense strategies against data poisoning have highlighted the effectiveness of aggregation schemes in achieving certified poisoning robustness. Here we focus on Deep Partition Aggregation, a representative aggregation defense, and assess its practical aspects, including efficiency, performance, and robustness.
  arXiv  Detail & Related papers  (2023-06-28T17:59:35Z)
• Data Poisoning Attack Aiming the Vulnerability of Continual Learning [25.480762565632332]
  We present a simple task-specific data poisoning attack that can be used in the learning process of a new task. We experiment with the attack on the two representative regularization-based continual learning methods.
  arXiv  Detail & Related papers  (2022-11-29T02:28:05Z)
• SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification [24.053704318868043]
  In model poisoning attacks, the attacker reduces the model's performance on targeted sub-tasks by uploading "poisoned" updates. We introduce algoname, a novel defense that uses global top-k update sparsification and device-level clipping gradient to mitigate model poisoning attacks.
  arXiv  Detail & Related papers  (2021-12-12T16:34:52Z)
• Policy Smoothing for Provably Robust Reinforcement Learning [109.90239627115336]
  We study the provable robustness of reinforcement learning against norm-bounded adversarial perturbations of the inputs. We generate certificates that guarantee that the total reward obtained by the smoothed policy will not fall below a certain threshold under a norm-bounded adversarial of perturbation the input.
  arXiv  Detail & Related papers  (2021-06-21T21:42:08Z)
• Learning and Certification under Instance-targeted Poisoning [49.55596073963654]
  We study PAC learnability and certification under instance-targeted poisoning attacks. We show that when the budget of the adversary scales sublinearly with the sample complexity, PAC learnability and certification are achievable. We empirically study the robustness of K nearest neighbour, logistic regression, multi-layer perceptron, and convolutional neural network on real data sets.
  arXiv  Detail & Related papers  (2021-05-18T17:48:15Z)
• Provable Defense Against Delusive Poisoning [64.69220849669948]
  We show that adversarial training can be a principled defense method against delusive poisoning. This implies that adversarial training can be a principled defense method against delusive poisoning.
  arXiv  Detail & Related papers  (2021-02-09T09:19:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.