ACIL: Analytic Class-Incremental Learning with Absolute Memorization and
Privacy Protection
- URL: http://arxiv.org/abs/2205.14922v1
- Date: Mon, 30 May 2022 08:41:56 GMT
- Title: ACIL: Analytic Class-Incremental Learning with Absolute Memorization and
Privacy Protection
- Authors: Huiping Zhuang, Zhenyu Weng, Renchunzi Xie, Kar-Ann Toh, Zhiping Lin
- Abstract summary: Class-incremental learning (CIL) learns a classification model with training data of different classes arising progressively.
Inspired by linear learning formulations, we propose an analytic class-incremental learning (ACIL) with absolute memorization of past knowledge.
- Score: 24.971575257217715
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Class-incremental learning (CIL) learns a classification model with training
data of different classes arising progressively. Existing CIL either suffers
from serious accuracy loss due to catastrophic forgetting, or invades data
privacy by revisiting used exemplars. Inspired by linear learning formulations,
we propose an analytic class-incremental learning (ACIL) with absolute
memorization of past knowledge while avoiding breaching of data privacy (i.e.,
without storing historical data). The absolute memorization is demonstrated in
the sense that class-incremental learning using ACIL given present data would
give identical results to that from its joint-learning counterpart which
consumes both present and historical samples. This equality is theoretically
validated. Data privacy is ensured since no historical data are involved during
the learning process. Empirical validations demonstrate ACIL's competitive
accuracy performance with near-identical results for various incremental task
settings (e.g., 5-50 phases). This also allows ACIL to outperform the
state-of-the-art methods for large-phase scenarios (e.g., 25 and 50 phases).
Related papers
- AOCIL: Exemplar-free Analytic Online Class Incremental Learning with Low Time and Resource Consumption [28.772554281694166]
Online Class Incremental Learning (OCIL) aims to train the model in a task-by-task manner, where data arrive in mini-batches at a time while previous data are not accessible.
To address this, replay-based methods show competitive results but invade data privacy, while exemplar-free methods protect data privacy but struggle for accuracy.
We propose an exemplar-free approach -- Analytic Online Class Incremental Learning (AOCIL)
arXiv Detail & Related papers (2024-03-23T07:39:13Z) - Memory Consistency Guided Divide-and-Conquer Learning for Generalized
Category Discovery [56.172872410834664]
Generalized category discovery (GCD) aims at addressing a more realistic and challenging setting of semi-supervised learning.
We propose a Memory Consistency guided Divide-and-conquer Learning framework (MCDL)
Our method outperforms state-of-the-art models by a large margin on both seen and unseen classes of the generic image recognition.
arXiv Detail & Related papers (2024-01-24T09:39:45Z) - Enhancing Consistency and Mitigating Bias: A Data Replay Approach for
Incremental Learning [100.7407460674153]
Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks.
To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks.
However, it is not expected in practice considering the memory constraint or data privacy issue.
As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
arXiv Detail & Related papers (2024-01-12T12:51:12Z) - Learning to Unlearn: Instance-wise Unlearning for Pre-trained
Classifiers [71.70205894168039]
We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model.
We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
arXiv Detail & Related papers (2023-01-27T07:53:50Z) - Forget Unlearning: Towards True Data-Deletion in Machine Learning [18.656957502454592]
We show that unlearning is not equivalent to data deletion and does not guarantee the "right to be forgotten"
We propose an accurate, computationally efficient, and secure data-deletion machine learning algorithm in the online setting.
arXiv Detail & Related papers (2022-10-17T10:06:11Z) - Continual Learning For On-Device Environmental Sound Classification [63.81276321857279]
We propose a simple and efficient continual learning method for on-device environmental sound classification.
Our method selects the historical data for the training by measuring the per-sample classification uncertainty.
arXiv Detail & Related papers (2022-07-15T12:13:04Z) - Queried Unlabeled Data Improves and Robustifies Class-Incremental
Learning [133.39254981496146]
Class-incremental learning (CIL) suffers from the notorious dilemma between learning newly added classes and preserving previously learned class knowledge.
We propose to leverage "free" external unlabeled data querying in continual learning.
We show queried unlabeled data can continue to benefit, and seamlessly extend CIL-QUD into its robustified versions.
arXiv Detail & Related papers (2022-06-15T22:53:23Z) - Quantifying and Mitigating Privacy Risks of Contrastive Learning [4.909548818641602]
We perform the first privacy analysis of contrastive learning through the lens of membership inference and attribute inference.
Our results show that contrastive models are less vulnerable to membership inference attacks but more vulnerable to attribute inference attacks compared to supervised models.
To remedy this situation, we propose the first privacy-preserving contrastive learning mechanism, namely Talos.
arXiv Detail & Related papers (2021-02-08T11:38:11Z) - ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for
Semi-supervised Continual Learning [52.831894583501395]
Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications.
We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN)
We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
arXiv Detail & Related papers (2021-01-02T09:04:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.