Tracing and Reversing Rank-One Model Edits

• URL: http://arxiv.org/abs/2505.20819v1
• Date: Tue, 27 May 2025 07:27:01 GMT
• Title: Tracing and Reversing Rank-One Model Edits
• Authors: Paul Youssef, Zhixue Zhao, Christin Seifert, Jörg Schlötterer,
• Abstract summary: This work investigates the traceability and reversibility of knowledge edits, focusing on the widely used Rank-One Model Editing (ROME) method.<n>We show that ROME introduces distinctive distributional patterns in the edited weight matrices, which can serve as effective signals for locating the edited weights.<n>We propose a method to infer the edited object entity directly from the modified weights, without access to the editing prompt, achieving over 95% accuracy.
• Score: 5.260519479124422
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Knowledge editing methods (KEs) are a cost-effective way to update the factual content of large language models (LLMs), but they pose a dual-use risk. While KEs are beneficial for updating outdated or incorrect information, they can be exploited maliciously to implant misinformation or bias. In order to defend against these types of malicious manipulation, we need robust techniques that can reliably detect, interpret, and mitigate adversarial edits. This work investigates the traceability and reversibility of knowledge edits, focusing on the widely used Rank-One Model Editing (ROME) method. We first show that ROME introduces distinctive distributional patterns in the edited weight matrices, which can serve as effective signals for locating the edited weights. Second, we show that these altered weights can reliably be used to predict the edited factual relation, enabling partial reconstruction of the modified fact. Building on this, we propose a method to infer the edited object entity directly from the modified weights, without access to the editing prompt, achieving over 95% accuracy. Finally, we demonstrate that ROME edits can be reversed, recovering the model's original outputs with $\geq$ 80% accuracy. Our findings highlight the feasibility of detecting, tracing, and reversing edits based on the edited weights, offering a robust framework for safeguarding LLMs against adversarial manipulations.

Related papers

• How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Models [7.342540592387184]
  We investigate the interaction between model editing and fine-tuning in the context of T2I diffusion models.<n>Our findings reveal a trend: edits generally fail to persist through fine-tuning, even when fine-tuning is tangential or unrelated to the edits.<n>These findings highlight the need for more robust techniques to ensure reliable long-term control and alignment of deployed AI systems.
  arXiv  Detail & Related papers  (2025-06-23T09:10:29Z)
• MEMOIR: Lifelong Model Editing with Minimal Overwrite and Informed Retention for LLMs [82.34547399693966]
  Existing methods for lifelong model editing compromise generalization, interfere with past edits, or fail to scale to long editing sequences.<n>We propose MEMOIR, a novel scalable framework that injects knowledge through a residual memory.<n>MeMOIR confines each edit to a distinct subset of the memory parameters, minimizing interference among edits.
  arXiv  Detail & Related papers  (2025-06-09T16:16:42Z)
• REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing [42.89229070245538]
  We introduce REACT, a framework for precise and controllable knowledge editing.<n>In the initial phase, we utilize tailored stimuli to extract latent factual representations.<n>In the second phase, we apply controllable perturbations to hidden states using the obtained vector with a magnitude scalar.
  arXiv  Detail & Related papers  (2025-05-25T01:57:06Z)
• K-Edit: Language Model Editing with Contextual Knowledge Awareness [71.73747181407323]
  Knowledge-based model editing enables precise modifications to the weights of large language models.<n>We present K-Edit, an effective approach to generating contextually consistent knowledge edits.
  arXiv  Detail & Related papers  (2025-02-15T01:35:13Z)
• ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA [55.697627106315004]
  Large language models (LLMs) require model editing to efficiently update specific knowledge within them and avoid factual errors.<n>Previous approaches manage sequential edits by freezing original parameters and discretely allocating new parameters for each knowledge update.<n>We propose ELDER, a novel approach to create a continuous association between data and adapters.
  arXiv  Detail & Related papers  (2024-08-19T02:27:00Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering [17.20346072074533]
  Recent model editing is a promising technique for efficiently updating a small amount of knowledge of large language models.<n>We propose a detachable and expandable Subject Word Embedding Altering (SWEA) framework, which finds the editing embeddings through token-level matching.<n>We demonstrate the overall state-of-the-art (SOTA) performance of SWEA$oplus$OS on the CounterFact and zsRE datasets.
  arXiv  Detail & Related papers  (2024-01-31T13:08:45Z)
• Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue [122.20016030723043]
  We evaluate the side effects of model editing on large language models (LLMs) Our analysis reveals that the side effects are caused by model editing altering the original model weights excessively. To mitigate this, a method named RECT is proposed to regularize the edit update weights.
  arXiv  Detail & Related papers  (2024-01-09T18:03:15Z)
• Memory-Based Model Editing at Scale [102.28475739907498]
  Existing model editors struggle to accurately model an edit's intended scope. We propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC) SERAC stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed.
  arXiv  Detail & Related papers  (2022-06-13T23:40:34Z)
• A Structural Model for Contextual Code Changes [20.185486717922615]
  Given a code snippet that is partially edited, our goal is to predict a completion of the edit for the rest of the snippet. Our model achieves a 28% relative gain over state-of-the-art sequential models and 2x higher accuracy than syntactic models that learn to generate the edited code.
  arXiv  Detail & Related papers  (2020-05-27T07:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.