The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing

• URL: http://arxiv.org/abs/2603.20795v1
• Date: Sat, 21 Mar 2026 12:40:15 GMT
• Title: The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
• Abstract summary: Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE)<n>We take a mechanistic view of KE using neuron-level knowledge attribution (NLKA)<n>Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact.<n>Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions.
• Score: 52.315369633116255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE). However, it remains unclear how edits are implemented inside the model once applied. In this work, we take a mechanistic view of KE using neuron-level knowledge attribution (NLKA). Unlike prior work that focuses on pre-edit causal tracing and localization, we use post-edit attribution -- contrasting successful and failed edits -- to isolate the computations that shift when an edit succeeds. Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact. Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions without modifying model weights. On CounterFact and Popular, MEGA achieves strong editing performance across KE metrics on GPT2-XL and LLaMA2-7B. Overall, our results elevate post-edit attribution from analysis to engineering signal: by pinpointing where and how edits take hold, it powers MEGA to deliver reliable, architecture-agnostic knowledge edits.

Related papers

• MetaKE: Meta-learning Aligned Knowledge Editing via Bi-level Optimization [11.430206422495829]
  State-of-the-art methods suffer from an open-loop control mismatch.<n>We identify a critical "Semantic-Execution Disconnect"<n>We propose MetaKE, a new framework that reframes KE as a bi-level optimization problem.
  arXiv  Detail & Related papers  (2026-03-13T05:47:00Z)
• Are We Evaluating the Edit Locality of LLM Model Editing Properly? [68.441768731381]
  We find that existing specificity evaluation protocols are inadequate for this purpose.<n>Existing specificity metrics are weakly correlated with the strength of specificity regularizers.<n>We also find that current metrics lack sufficient sensitivity, rendering them ineffective at distinguishing the specificity performance of different methods.
  arXiv  Detail & Related papers  (2026-01-24T07:07:21Z)
• Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs [71.8547241246169]
  We introduce CogEdit, a novel benchmark designed to evaluate MLLMs' meta-cognitive knowledge editing abilities.<n>We propose MIND, a framework that constructs a meta-knowledge memory for self-awareness, employs game-theoretic interactions to monitor knowledge activation, and incorporates label refinement for noise-robust updates.
  arXiv  Detail & Related papers  (2025-09-06T13:26:04Z)
• Aligning Language Models with Real-time Knowledge Editing [11.503574001763246]
  We introduce CRAFT, an ever-evolving real-world benchmark for knowledge editing.<n>It features well-designed paired edits for composite reasoning, and evaluates models on alias portability and temporal and common-sense locality.<n> Towards flexible real-time editing, we propose KEDAS, a novel paradigm of knowledge editing alignment featuring diverse edit augmentation and self-adaptive post-alignment inference.
  arXiv  Detail & Related papers  (2025-08-02T10:25:36Z)
• Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editing [72.8373875453882]
  knowledge editing (KE) has emerged as a promising approach to update specific facts in Large Language Models (LLMs) without the need for full retraining.<n>We propose a novel framework called MedEditBench to rigorously evaluate the effectiveness of existing KE methods in the medical domain.<n>Our findings indicate that current KE methods result in only superficial memorization of the injected information, failing to generalize to new scenarios.
  arXiv  Detail & Related papers  (2025-06-04T02:14:43Z)
• Tracing and Reversing Rank-One Model Edits [5.260519479124422]
  This work investigates the traceability and reversibility of knowledge edits, focusing on the widely used Rank-One Model Editing (ROME) method.<n>We show that ROME introduces distinctive distributional patterns in the edited weight matrices, which can serve as effective signals for locating the edited weights.<n>We propose a method to infer the edited object entity directly from the modified weights, without access to the editing prompt, achieving over 95% accuracy.
  arXiv  Detail & Related papers  (2025-05-27T07:27:01Z)
• Uncovering Overfitting in Large Language Model Editing [35.55260822503773]
  We identify and investigate the phenomenon of Editing Overfit, where edited models assign disproportionately high probabilities to the edit target.<n>We propose Learn the Inference (LTI), which introduce a Multi-stage Inference Constraint module to guide the edited models in recalling new knowledge.
  arXiv  Detail & Related papers  (2024-10-10T11:09:00Z)
• MEMoE: Enhancing Model Editing with Mixture of Experts Adaptors [30.831866499812925]
  MEMoE is a model editing adapter utilizing a Mixture of Experts (MoE) architecture with a knowledge anchor routing strategy. We show the superiority of our approach over both batch editing and sequential batch editing tasks.
  arXiv  Detail & Related papers  (2024-05-29T13:49:44Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering [17.20346072074533]
  Recent model editing is a promising technique for efficiently updating a small amount of knowledge of large language models.<n>We propose a detachable and expandable Subject Word Embedding Altering (SWEA) framework, which finds the editing embeddings through token-level matching.<n>We demonstrate the overall state-of-the-art (SOTA) performance of SWEA$oplus$OS on the CounterFact and zsRE datasets.
  arXiv  Detail & Related papers  (2024-01-31T13:08:45Z)
• Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue [122.20016030723043]
  We evaluate the side effects of model editing on large language models (LLMs) Our analysis reveals that the side effects are caused by model editing altering the original model weights excessively. To mitigate this, a method named RECT is proposed to regularize the edit update weights.
  arXiv  Detail & Related papers  (2024-01-09T18:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.