Machine Unlearning in Generative AI: A Survey

• URL: http://arxiv.org/abs/2407.20516v1
• Date: Tue, 30 Jul 2024 03:26:09 GMT
• Title: Machine Unlearning in Generative AI: A Survey
• Authors: Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, Meng Jiang,
• Abstract summary: Generative AI technologies have been deployed in many places, such as (multimodal) large language models and vision generative models. New machine unlearning (MU) techniques are being developed to reduce or eliminate undesirable knowledge.
• Score: 19.698620794387338
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Generative AI technologies have been deployed in many places, such as (multimodal) large language models and vision generative models. Their remarkable performance should be attributed to massive training data and emergent reasoning abilities. However, the models would memorize and generate sensitive, biased, or dangerous information originated from the training data especially those from web crawl. New machine unlearning (MU) techniques are being developed to reduce or eliminate undesirable knowledge and its effects from the models, because those that were designed for traditional classification tasks could not be applied for Generative AI. We offer a comprehensive survey on many things about MU in Generative AI, such as a new problem formulation, evaluation methods, and a structured discussion on the advantages and limitations of different kinds of MU techniques. It also presents several critical challenges and promising directions in MU research. A curated list of readings can be found: https://github.com/franciscoliu/GenAI-MU-Reading.

Related papers

• Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities [89.40778301238642]
  Model merging is an efficient empowerment technique in the machine learning community. There is a significant gap in the literature regarding a systematic and thorough review of these techniques.
  arXiv  Detail & Related papers  (2024-08-14T16:58:48Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
  Iterative Contrastive Unlearning (ICU) framework consists of three core components. A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss. A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal. And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• Procedural Content Generation via Generative Artificial Intelligence [1.437446768735628]
  generative artificial intelligence (AI) saw a significant increase in interest in the mid-2010s. generative AI is effective for PCG, but building high-performance AI requires vast amounts of training data. For PCG research to advance further, issues related to limited training data must be overcome.
  arXiv  Detail & Related papers  (2024-07-12T06:03:38Z)
• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning [53.90744622542961]
  Reasoning in mathematical domains remains a significant challenge for small language models (LMs) We introduce a new method that exploits existing mathematical problem datasets with diverse annotation styles. Experimental results show that our strategy enables a LLaMA-7B model to outperform prior approaches.
  arXiv  Detail & Related papers  (2023-07-16T05:41:53Z)
• Learn to Unlearn: A Survey on Machine Unlearning [29.077334665555316]
  This article presents a review of recent machine unlearning techniques, verification mechanisms, and potential attacks. We highlight emerging challenges and prospective research directions. We aim for this paper to provide valuable resources for integrating privacy, equity, andresilience into ML systems.
  arXiv  Detail & Related papers  (2023-05-12T14:28:02Z)
• Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy [17.535417441295074]
  Machine unlearning (MU) is gaining increasing attention due to the need to remove or modify predictions made by machine learning (ML) models. This paper presents a comprehensive survey of MU, covering current state-of-the-art techniques and approaches. The paper also highlights the challenges that need to be addressed, including attack sophistication, standardization, transferability, interpretability, and resource constraints.
  arXiv  Detail & Related papers  (2023-05-10T12:02:18Z)
• Learnware: Small Models Do Big [69.88234743773113]
  The prevailing big model paradigm, which has achieved impressive results in natural language processing and computer vision applications, has not yet addressed those issues, whereas becoming a serious source of carbon emissions. This article offers an overview of the learnware paradigm, which attempts to enable users not need to build machine learning models from scratch, with the hope of reusing small models to do things even beyond their original purposes.
  arXiv  Detail & Related papers  (2022-10-07T15:55:52Z)
• A Survey of Machine Unlearning [56.017968863854186]
  Recent regulations now require that, on request, private information about a user must be removed from computer systems. ML models often remember' the old data. Recent works on machine unlearning have not been able to completely solve the problem.
  arXiv  Detail & Related papers  (2022-09-06T08:51:53Z)
• Novel Applications for VAE-based Anomaly Detection Systems [5.065947993017157]
  Deep generative modeling (DGM) can create novel and unseen data, starting from a given data set. As the technology shows promising applications, many ethical issues also arise. Research indicates different biases affect deep learning models, leading to social issues such as misrepresentation.
  arXiv  Detail & Related papers  (2022-04-26T20:30:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.