TruVRF: Towards Triple-Granularity Verification on Machine Unlearning

• URL: http://arxiv.org/abs/2408.06063v1
• Date: Mon, 12 Aug 2024 11:29:54 GMT
• Title: TruVRF: Towards Triple-Granularity Verification on Machine Unlearning
• Abstract summary: TruVRF is a non-invasive unlearning verification framework operating at class-, volume-, and sample-level granularities. It includes three Unlearning-Metrics designed to detect different types of dishonest servers: Neglecting, Lazy, and Deceiving. We show TruVRF's robust performance, with over 90% accuracy for Metrics I and III, and a 4.8% to 8.2% inference deviation for Metric II.
• Score: 4.3667223256713745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The concept of the right to be forgotten has led to growing interest in machine unlearning, but reliable validation methods are lacking, creating opportunities for dishonest model providers to mislead data contributors. Traditional invasive methods like backdoor injection are not feasible for legacy data. To address this, we introduce TruVRF, a non-invasive unlearning verification framework operating at class-, volume-, and sample-level granularities. TruVRF includes three Unlearning-Metrics designed to detect different types of dishonest servers: Neglecting, Lazy, and Deceiving. Unlearning-Metric-I checks class alignment, Unlearning-Metric-II verifies sample count, and Unlearning-Metric-III confirms specific sample deletion. Evaluations on three datasets show TruVRF's robust performance, with over 90% accuracy for Metrics I and III, and a 4.8% to 8.2% inference deviation for Metric II. TruVRF also demonstrates generalizability and practicality across various conditions and with state-of-the-art unlearning frameworks like SISA and Amnesiac Unlearning.

Related papers

• RULER: Representation-Level Verification of Machine Unlearning [0.17188280334580197]
  We introduce RULER, a set of representation-level verification metrics.<n>M2 measures whether forget-set records occupy the same representational position as in a model retrained without them.<n>M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining.
  arXiv  Detail & Related papers  (2026-05-26T18:41:48Z)
• Not Every Subject Should Stay: Machine Unlearning for Noisy Engagement Recognition [53.005382593686356]
  Engagement recognition datasets are typically subject-indexed and often contain noisy, subjective supervision.<n>We study this setting through subject-level machine unlearning as a post-hoc sanitization mechanism for engagement recognition.
  arXiv  Detail & Related papers  (2026-05-06T10:03:06Z)
• Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning [59.19460954480119]
  We study whether forgotten knowledge originates from pretraining or supervised fine-tuning.<n>Our experiments show that pretrained and SFT models respond differently to unlearning.
  arXiv  Detail & Related papers  (2026-02-23T08:58:48Z)
• Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance [51.532841645285835]
  We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution.<n>We show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution.<n>We introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure.
  arXiv  Detail & Related papers  (2026-02-10T19:08:40Z)
• REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMs [0.1784233255402269]
  Machine unlearning aims to remove the influence of specific training data from a model without requiring full retraining.<n>We propose REMIND, a novel evaluation method aiming to detect the subtle remaining influence of unlearned data.<n>We show that unlearned data yield flatter, less steep loss landscapes, while retained or unrelated data exhibit sharper, more volatile patterns.
  arXiv  Detail & Related papers  (2025-11-06T09:58:19Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• Targeted Forgetting of Image Subgroups in CLIP Models [30.78624907082701]
  Foundation models (FMs) such as CLIP have demonstrated impressive zero-shot performance across various tasks.<n>They often inherit harmful or unwanted knowledge from noisy internet-sourced datasets.<n>Existing model unlearning methods either rely on access to pre-trained datasets or focus on coarse-grained unlearning.<n>We propose a novel three-stage approach that progressively unlearns targeted knowledge while mitigating over-forgetting.
  arXiv  Detail & Related papers  (2025-06-03T17:50:03Z)
• UniErase: Unlearning Token as a Universal Erasure Primitive for Language Models [54.75551043657238]
  We introduce UniErase, a novel unlearning paradigm that employs learnable parametric suffix (unlearning token) to steer language models toward targeted forgetting behaviors.<n>UniErase achieves state-of-the-art (SOTA) performance across batch, sequential, and precise unlearning under fictitious and real-world knowledge settings.
  arXiv  Detail & Related papers  (2025-05-21T15:53:28Z)
• Instance-Level Data-Use Auditing of Visual ML Models [47.369572284751285]
  Growing trend of legal disputes over the unauthorized use of data in machine learning (ML) systems highlights the need for reliable data-use auditing mechanisms. We present the first proactive instance-level data-use auditing method designed to enable data owners to audit the use of their individual data instances in ML models.
  arXiv  Detail & Related papers  (2025-03-28T13:28:57Z)
• Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models [70.78205685001168]
  We investigate knowledge forgetting in large language models with a focus on its generalisation.<n> UGBench is the first benchmark specifically designed to assess the unlearning of in-scope implicit knowledge.<n>We propose PerMU, a novel probability-based unlearning paradigm.
  arXiv  Detail & Related papers  (2025-02-27T11:03:33Z)
• Neural Corrective Machine Unranking [3.2340528215722553]
  We formalise corrective unranking and propose a novel teacher-student framework, Corrective unRanking Distillation (CuRD) CuRD facilitates forgetting by adjusting the (trained) neural IR model such that its output relevance scores of to-be-forgotten samples mimic those of low-ranking, non-retrievable samples. Experiments with forget set sizes from 1 % and 20 % of the training dataset demonstrate that CuRD outperforms seven state-of-the-art baselines in terms of forgetting and correction.
  arXiv  Detail & Related papers  (2024-11-13T12:19:46Z)
• RESTOR: Knowledge Recovery through Machine Unlearning [71.75834077528305]
  Large language models trained on web-scale corpora can memorize undesirable datapoints. Many machine unlearning methods have been proposed that aim to 'erase' these datapoints from trained models. We propose the RESTOR framework for machine unlearning based on the following dimensions.
  arXiv  Detail & Related papers  (2024-10-31T20:54:35Z)
• Alignment Calibration: Machine Unlearning for Contrastive Learning under Auditing [33.418062986773606]
  We first propose the framework of Machine Unlearning for Contrastive learning (MUC) and adapting existing methods. We observe that several methods are mediocre unlearners and existing auditing tools may not be sufficient for data owners to validate the unlearning effects in contrastive learning. We propose a novel method called Alignment (AC) by explicitly considering the properties of contrastive learning and optimizing towards novel metrics to easily verify unlearning.
  arXiv  Detail & Related papers  (2024-06-05T19:55:45Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• To Be Forgotten or To Be Fair: Unveiling Fairness Implications of Machine Unlearning Methods [31.83413902054534]
  We present the first study on machine unlearning methods to reveal their fairness implications. Under non-uniform data deletion, SISA leads to better fairness compared with ORTR and AmnesiacML, while initial training and uniform data deletion do not necessarily affect the fairness of all three methods.
  arXiv  Detail & Related papers  (2023-02-07T09:48:29Z)
• Verifiable and Provably Secure Machine Unlearning [44.142771334058715]
  Machine unlearning aims to remove points from the training dataset of a machine learning model after training. We present the first cryptographic definition of verifiable unlearning to formally capture the guarantees of an unlearning system. We implement the protocol for three different unlearning techniques and validate its feasibility for linear regression, logistic regression, and neural networks.
  arXiv  Detail & Related papers  (2022-10-17T14:19:52Z)
• Fraud Detection Using Optimized Machine Learning Tools Under Imbalance Classes [0.304585143845864]
  Fraud detection with smart versions of machine learning (ML) tools is essential to assure safety. We investigate four state-of-the-art ML techniques, namely, logistic regression, decision trees, random forest, and extreme gradient boost. For phishing website URLs and credit card fraud transaction datasets, the results indicate that extreme gradient boost trained on the original data shows trustworthy performance.
  arXiv  Detail & Related papers  (2022-09-04T15:30:23Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)
• Certifiable Machine Unlearning for Linear Models [1.484852576248587]
  Machine unlearning is the task of updating machine learning (ML) models after a subset of the training data they were trained on is deleted. We present an experimental study of the three state-of-the-art approximate unlearning methods for linear models.
  arXiv  Detail & Related papers  (2021-06-29T05:05:58Z)
• SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks [81.64530401885476]
  We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties. Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns. We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
  arXiv  Detail & Related papers  (2020-10-19T09:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.