Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?

• URL: http://arxiv.org/abs/2406.19354v1
• Date: Thu, 27 Jun 2024 17:33:03 GMT
• Title: Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?
• Authors: Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel-Eskin, Mohit Bansal,
• Abstract summary: This paper critiques the standard formulation of the model editing problem and proposes a formal testbed for model editing research. We first describe 12 open problems with model editing, based on challenges with (1) defining the problem, (2) developing benchmarks, and (3) assuming LLMs have editable beliefs in the first place. Next, we introduce a semi-synthetic dataset for model editing based on Wikidata, where we can evaluate edits against labels given by an idealized Bayesian agent.
• Score: 61.68363765350178
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The model editing problem concerns how language models should learn new facts about the world over time. While empirical research on model editing has drawn widespread attention, the conceptual foundations of model editing remain shaky -- perhaps unsurprisingly, since model editing is essentially belief revision, a storied problem in philosophy that has eluded succinct solutions for decades. Model editing nonetheless demands a solution, since we need to be able to control the knowledge within language models. With this goal in mind, this paper critiques the standard formulation of the model editing problem and proposes a formal testbed for model editing research. We first describe 12 open problems with model editing, based on challenges with (1) defining the problem, (2) developing benchmarks, and (3) assuming LLMs have editable beliefs in the first place. Many of these challenges are extremely difficult to address, e.g. determining far-reaching consequences of edits, labeling probabilistic entailments between facts, and updating beliefs of agent simulators. Next, we introduce a semi-synthetic dataset for model editing based on Wikidata, where we can evaluate edits against labels given by an idealized Bayesian agent. This enables us to say exactly how belief revision in language models falls short of a desirable epistemic standard. We encourage further research exploring settings where such a gold standard can be compared against. Our code is publicly available at: https://github.com/peterbhase/LLM-belief-revision

Related papers

• Should We Really Edit Language Models? On the Evaluation of Edited Language Models [15.63231238452797]
  Existing editing methods lead to inevitable performance deterioration on general benchmarks. Instruction-tuned models are more robust to editing, showing less performance drop on general knowledge after editing. Our findings indicate that current editing methods are only suitable for small-scale knowledge updates within language models.
  arXiv  Detail & Related papers  (2024-10-24T14:36:48Z)
• Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization [48.07144492109635]
  Large language models need to be updated regularly. Model editing is challenging as it might also affect knowledge that is unrelated to the new data. We propose SAUL, a streamlined model editing method that uses sentence concatenation with augmented random facts for generation regularization.
  arXiv  Detail & Related papers  (2024-10-03T12:28:13Z)
• "Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models [17.77377809345631]
  We investigate how model editing methods unexpectedly amplify model biases post-edit. Specifically, we focus on biases with respect to demographic attributes such as race, geographic origin, and gender. We find that edited models exhibit, to various degrees, more biased behavior as they become less confident in attributes for Asian, African, and South American subjects.
  arXiv  Detail & Related papers  (2024-02-29T23:11:55Z)
• Potential and Challenges of Model Editing for Social Debiasing [20.186721346693577]
  Large language models (LLMs) trained on vast corpora suffer from inevitable stereotype biases. Mitigating these biases with fine-tuning could be both costly and data-hungry. Model editing methods, which focus on modifying LLMs in a post-hoc manner, are of great potential to address debiasing.
  arXiv  Detail & Related papers  (2024-02-21T01:35:26Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• DUnE: Dataset for Unified Editing [3.7346004746366384]
  We introduce DUnE-an editing benchmark where edits are natural language sentences. We show that retrieval-augmented language modeling can outperform specialized editing techniques.
  arXiv  Detail & Related papers  (2023-11-27T18:56:14Z)
• Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models [68.03946716358335]
  We find that we can change how a fact is stored in a model by editing weights that are in a different location than where existing methods suggest that the fact is stored. This is surprising because we would expect that localizing facts to specific model parameters would tell us where to manipulate knowledge in models. Our results suggest, counterintuitively, that better mechanistic understanding of how pretrained language models work may not always translate to insights about how to best change their behavior.
  arXiv  Detail & Related papers  (2023-01-10T21:26:08Z)
• Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors [53.819805242367345]
  We propose GRACE, a lifelong model editing method, which implements spot-fixes on streaming errors of a deployed model. GRACE writes new mappings into a pre-trained model's latent space, creating a discrete, local codebook of edits without altering model weights. Our experiments on T5, BERT, and GPT models show GRACE's state-of-the-art performance in making and retaining edits, while generalizing to unseen inputs.
  arXiv  Detail & Related papers  (2022-11-20T17:18:22Z)
• Memory-Based Model Editing at Scale [102.28475739907498]
  Existing model editors struggle to accurately model an edit's intended scope. We propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC) SERAC stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed.
  arXiv  Detail & Related papers  (2022-06-13T23:40:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.