Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization

• URL: http://arxiv.org/abs/2410.12949v1
• Date: Wed, 16 Oct 2024 18:35:02 GMT
• Title: Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization
• Authors: Phillip Guo, Aaquib Syed, Abhay Sheshadri, Aidan Ewart, Gintare Karolina Dziugaite,
• Abstract summary: Methods for knowledge editing and unlearning in large language models seek to edit or remove undesirable knowledge without compromising performance. We find a stark difference in unlearning and edit robustness when training components localized by different methods.
• Score: 10.944365976254442
• License:
• Abstract: Methods for knowledge editing and unlearning in large language models seek to edit or remove undesirable knowledge or capabilities without compromising general language modeling performance. This work investigates how mechanistic interpretability -- which, in part, aims to identify model components (circuits) associated to specific interpretable mechanisms that make up a model capability -- can improve the precision and effectiveness of editing and unlearning. We find a stark difference in unlearning and edit robustness when training components localized by different methods. We highlight an important distinction between methods that localize components based primarily on preserving outputs, and those finding high level mechanisms with predictable intermediate states. In particular, localizing edits/unlearning to components associated with the lookup-table mechanism for factual recall 1) leads to more robust edits/unlearning across different input/output formats, and 2) resists attempts to relearn the unwanted information, while also reducing unintended side effects compared to baselines, on both a sports facts dataset and the CounterFact dataset across multiple models. We also find that certain localized edits disrupt the latent knowledge in the model more than any other baselines, making unlearning more robust to various attacks.

Related papers

• RESTOR: Knowledge Recovery through Machine Unlearning [71.75834077528305]
  Large language models trained on web-scale corpora can memorize undesirable datapoints. Many machine unlearning methods have been proposed that aim to 'erase' these datapoints from trained models. We propose the RESTOR framework for machine unlearning based on the following dimensions.
  arXiv  Detail & Related papers  (2024-10-31T20:54:35Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
  Iterative Contrastive Unlearning (ICU) framework consists of three core components. A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss. A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal. And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• How Truncating Weights Improves Reasoning in Language Models [49.80959223722325]
  We study how certain global associations tend to be stored in specific weight components or Transformer blocks. We analyze how this arises during training, both empirically and theoretically.
  arXiv  Detail & Related papers  (2024-06-05T08:51:08Z)
• Federated Unlearning via Active Forgetting [24.060724751342047]
  We propose a novel federated unlearning framework based on incremental learning. Our framework differs from existing federated unlearning methods that rely on approximate retraining or data influence estimation.
  arXiv  Detail & Related papers  (2023-07-07T03:07:26Z)
• Knowledge-Infused Self Attention Transformers [11.008412414253662]
  Transformer-based language models have achieved impressive success in various natural language processing tasks. This paper introduces a systematic method for infusing knowledge into different components of a transformer-based model.
  arXiv  Detail & Related papers  (2023-06-23T13:55:01Z)
• Relational Local Explanations [11.679389861042]
  We develop a novel model-agnostic and permutation-based feature attribution algorithm based on relational analysis between input variables. We are able to gain a broader insight into machine learning model decisions and data.
  arXiv  Detail & Related papers  (2022-12-23T14:46:23Z)
• Change Detection for Local Explainability in Evolving Data Streams [72.4816340552763]
  Local feature attribution methods have become a popular technique for post-hoc and model-agnostic explanations. It is often unclear how local attributions behave in realistic, constantly evolving settings such as streaming and online applications. We present CDLEEDS, a flexible and model-agnostic framework for detecting local change and concept drift.
  arXiv  Detail & Related papers  (2022-09-06T18:38:34Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Knodle: Modular Weakly Supervised Learning with PyTorch [5.874587993411972]
  Knodle is a software framework for separating weak data annotations, powerful deep learning models, and methods for improving weakly supervised training. This modularization gives the training process access to fine-grained information such as data set characteristics, matches of rules, or elements of the deep learning model ultimately used for prediction.
  arXiv  Detail & Related papers  (2021-04-23T12:33:25Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.