Unlearning Targeted Information via Single Layer Unlearning Gradient

• URL: http://arxiv.org/abs/2407.11867v2
• Date: Thu, 5 Sep 2024 19:19:59 GMT
• Title: Unlearning Targeted Information via Single Layer Unlearning Gradient
• Authors: Zikui Cai, Yaoteng Tan, M. Salman Asif,
• Abstract summary: Unauthorized privacy-related computation is a significant concern for society. The EU's General Protection Regulation includes a "right to be forgotten" We propose Single Layer Unlearning Gradient (SLUG) to unlearn targeted information by updating targeted layers of a model.
• Score: 15.374381635334897
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unauthorized privacy-related and copyrighted content generation using generative-AI is becoming a significant concern for human society, raising ethical, legal, and privacy issues that demand urgent attention. The EU's General Data Protection Regulation (GDPR) include a "right to be forgotten," which allows individuals to request the deletion of their personal data. However, this primarily applies to data stored in traditional databases, not AI models. Recently, machine unlearning techniques have arise that attempt to eliminate the influence of sensitive content used during AI model training, but they often require extensive updates to the deployed systems and incur substantial computational costs. In this work, we propose a novel and efficient method called Single Layer Unlearning Gradient (SLUG), that can unlearn targeted information by updating targeted layers of a model using a one-time gradient computation. Our method is highly modular and enables the selective removal of multiple sensitive concepts, such as celebrity names and copyrighted content, from the generated outputs of widely used foundation models (e.g., CLIP) and generative models (e.g., Stable Diffusion). Broadly, our method ensures AI-generated content complies with privacy regulations and intellectual property laws, fostering responsible use of generative models, mitigating legal risks and promoting a trustworthy, socially responsible AI ecosystem.

Related papers

• Game-Theoretic Machine Unlearning: Mitigating Extra Privacy Leakage [12.737028324709609]
  Recent legislation obligates organizations to remove requested data and its influence from a trained model. We propose a game-theoretic machine unlearning algorithm that simulates the competitive relationship between unlearning performance and privacy protection.
  arXiv  Detail & Related papers  (2024-11-06T13:47:04Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
  Iterative Contrastive Unlearning (ICU) framework consists of three core components. A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss. A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal. And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• Generative Models are Self-Watermarked: Declaring Model Authentication through Re-Generation [17.88043926057354]
  verifying data ownership poses formidable challenges, particularly in cases of unauthorized reuse of generated data. Our work is dedicated to detecting data reuse from even an individual sample. We propose an explainable verification procedure that attributes data ownership through re-generation, and further amplifies these fingerprints in the generative models through iterative data re-generation.
  arXiv  Detail & Related papers  (2024-02-23T10:48:21Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• Your Room is not Private: Gradient Inversion Attack on Reinforcement Learning [47.96266341738642]
  Privacy emerges as a pivotal concern within the realm of embodied AI, as the robot accesses substantial personal information. This paper proposes an attack on the value-based algorithm and the gradient-based algorithm, utilizing gradient inversion to reconstruct states, actions, and supervision signals.
  arXiv  Detail & Related papers  (2023-06-15T16:53:26Z)
• A Survey of Machine Unlearning [56.017968863854186]
  Recent regulations now require that, on request, private information about a user must be removed from computer systems. ML models often remember' the old data. Recent works on machine unlearning have not been able to completely solve the problem.
  arXiv  Detail & Related papers  (2022-09-06T08:51:53Z)
• Disentangling private classes through regularization [8.72305226979945]
  We propose DisP, an approach for deep learning models disentangling the information related to some classes we desire to keep private. DisP is a regularization strategy de-correlating the features belonging to the same private class at training time, hiding the information of private classes membership. Our experiments on state-of-the-art deep learning models show the effectiveness of DisP.
  arXiv  Detail & Related papers  (2022-07-05T12:35:47Z)
• Representative & Fair Synthetic Data [68.8204255655161]
  We present a framework to incorporate fairness constraints into the self-supervised learning process. We generate a representative as well as fair version of the UCI Adult census data set. We consider representative & fair synthetic data a promising future building block to teach algorithms not on historic worlds, but rather on the worlds that we strive to live in.
  arXiv  Detail & Related papers  (2021-04-07T09:19:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.