Instance-Level Difficulty: A Missing Perspective in Machine Unlearning

• URL: http://arxiv.org/abs/2410.03043v2
• Date: Fri, 21 Feb 2025 04:44:56 GMT
• Title: Instance-Level Difficulty: A Missing Perspective in Machine Unlearning
• Authors: Hammad Rizwan, Mahtab Sarvmaili, Hassan Sajjad, Ga Wu,
• Abstract summary: We study the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis.<n>In particular, we summarize four factors that make unlearning a data point difficult.<n>We argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.
• Score: 13.052520843129363
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current research on deep machine unlearning primarily focuses on improving or evaluating the overall effectiveness of unlearning methods while overlooking the varying difficulty of unlearning individual training samples. As a result, the broader feasibility of machine unlearning remains under-explored. This paper studies the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis over various unlearning algorithms and datasets. In particular, we summarize four factors that make unlearning a data point difficult, and we empirically show that these factors are independent of a specific unlearning algorithm but only relevant to the target model and its training data. Given these findings, we argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.

Related papers

• When to Forget? Complexity Trade-offs in Machine Unlearning [23.507879460531264]
  Machine Unlearning (MU) aims at removing the influence of specific data points from a trained model. We analyze the efficiency of unlearning methods and establish the first upper and lower bounds on minimax times for this problem. We provide a phase diagram for the unlearning complexity ratio -- a novel metric that compares the computational cost of the best unlearning method to full model retraining.
  arXiv  Detail & Related papers  (2025-02-24T16:56:27Z)
• Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method [31.268301764230525]
  This work formalizes a metric to evaluate unlearning quality in generative models. We use it to assess the trade-offs between unlearning quality and performance. We further evaluate how example's memorization and difficulty affect unlearning under a classical gradient ascent-based approach.
  arXiv  Detail & Related papers  (2024-11-07T03:02:09Z)
• RESTOR: Knowledge Recovery through Machine Unlearning [71.75834077528305]
  Large language models trained on web-scale corpora can memorize undesirable datapoints. Many machine unlearning algorithms have been proposed that aim to erase' these datapoints. We propose the RESTOR framework for machine unlearning, which evaluates the ability of unlearning algorithms to perform targeted data erasure.
  arXiv  Detail & Related papers  (2024-10-31T20:54:35Z)
• Evaluating of Machine Unlearning: Robustness Verification Without Prior Modifications [15.257558809246524]
  Unlearning is a process enabling pre-trained models to remove the influence of specific training samples. Existing verification methods rely on machine learning attack techniques, such as membership inference attacks (MIAs) or backdoor attacks. We propose a novel verification scheme without any prior modifications, and can support verification on a much larger set.
  arXiv  Detail & Related papers  (2024-10-14T03:19:14Z)
• Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning [97.2995389188179]
  Recent research has begun to approach large language models (LLMs) unlearning via gradient ascent (GA) Despite their simplicity and efficiency, we suggest that GA-based methods face the propensity towards excessive unlearning. We propose several controlling methods that can regulate the extent of excessive unlearning.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• Machine Unlearning in Contrastive Learning [3.6218162133579694]
  We introduce a novel gradient constraint-based approach for training the model to effectively achieve machine unlearning. Our approach demonstrates proficient performance not only on contrastive learning models but also on supervised learning models.
  arXiv  Detail & Related papers  (2024-05-12T16:09:01Z)
• Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning [16.809644622465086]
  We conduct the first investigation to understand the extent to which machine unlearning can leak the confidential content of unlearned data. Under the Machine Learning as a Service setting, we propose unlearning inversion attacks that can reveal the feature and label information of an unlearned sample. The experimental results indicate that the proposed attack can reveal the sensitive information of the unlearned data.
  arXiv  Detail & Related papers  (2024-04-04T06:37:46Z)
• Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Approximate Unlearning Completeness [30.596695293390415]
  We introduce the task of Lifecycle Unlearning Commitment Management (LUCM) for approximate unlearning. We propose an efficient metric designed to assess the sample-level unlearning completeness. We show that this metric is able to serve as a tool for monitoring unlearning anomalies throughout the unlearning lifecycle.
  arXiv  Detail & Related papers  (2024-03-19T15:37:27Z)
• Machine Unlearning: A Survey [56.79152190680552]
  A special need has arisen where, due to privacy, usability, and/or the right to be forgotten, information about some specific samples needs to be removed from a model, called machine unlearning. This emerging technology has drawn significant interest from both academics and industry due to its innovation and practicality. No study has analyzed this complex topic or compared the feasibility of existing unlearning solutions in different kinds of scenarios. The survey concludes by highlighting some of the outstanding issues with unlearning techniques, along with some feasible directions for new research opportunities.
  arXiv  Detail & Related papers  (2023-06-06T10:18:36Z)
• Accelerating exploration and representation learning with offline pre-training [52.6912479800592]
  We show that exploration and representation learning can be improved by separately learning two different models from a single offline dataset. We show that learning a state representation using noise-contrastive estimation and a model of auxiliary reward can significantly improve the sample efficiency on the challenging NetHack benchmark.
  arXiv  Detail & Related papers  (2023-03-31T18:03:30Z)
• Frugal Reinforcement-based Active Learning [12.18340575383456]
  We propose a novel active learning approach for label-efficient training. The proposed method is iterative and aims at minimizing a constrained objective function that mixes diversity, representativity and uncertainty criteria. We also introduce a novel weighting mechanism based on reinforcement learning, which adaptively balances these criteria at each training iteration.
  arXiv  Detail & Related papers  (2022-12-09T14:17:45Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• Impact Learning: A Learning Method from Features Impact and Competition [1.3569491184708429]
  This paper introduced a new machine learning algorithm called impact learning. Impact learning is a supervised learning algorithm that can be consolidated in both classification and regression problems. It is prepared by the impacts of the highlights from the intrinsic rate of natural increase.
  arXiv  Detail & Related papers  (2022-11-04T04:56:35Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection [55.888835686183995]
  We propose a neural network-based meta-learning method for supervised anomaly detection. We experimentally demonstrate that the proposed method achieves better performance than existing anomaly detection and few-shot learning methods.
  arXiv  Detail & Related papers  (2021-03-01T01:43:04Z)
• Curriculum Learning: A Survey [65.31516318260759]
  Curriculum learning strategies have been successfully employed in all areas of machine learning. We construct a taxonomy of curriculum learning approaches by hand, considering various classification criteria. We build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm.
  arXiv  Detail & Related papers  (2021-01-25T20:08:32Z)
• Knowledge as Invariance -- History and Perspectives of Knowledge-augmented Machine Learning [69.99522650448213]
  Research in machine learning is at a turning point. Research interests are shifting away from increasing the performance of highly parameterized models to exceedingly specific tasks. This white paper provides an introduction and discussion of this emerging field in machine learning research.
  arXiv  Detail & Related papers  (2020-12-21T15:07:19Z)
• Provable Meta-Learning of Linear Representations [114.656572506859]
  We provide fast, sample-efficient algorithms to address the dual challenges of learning a common set of features from multiple, related tasks, and transferring this knowledge to new, unseen tasks. We also provide information-theoretic lower bounds on the sample complexity of learning these linear features.
  arXiv  Detail & Related papers  (2020-02-26T18:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.