Mechanistic Circuit-Based Knowledge Editing in Large Language Models

• URL: http://arxiv.org/abs/2604.05876v1
• Date: Tue, 07 Apr 2026 13:33:54 GMT
• Title: Mechanistic Circuit-Based Knowledge Editing in Large Language Models
• Abstract summary: MCircKE is a novel framework that enables a precise "map-and-adapt" editing procedure.<n>It first identifies the causal circuits responsible for a specific reasoning task.<n>It then surgically update parameters exclusively within this mapped circuit.
• Score: 13.74199057436712
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying Large Language Models (LLMs) in real-world dynamic environments raises the challenge of updating their pre-trained knowledge. While existing knowledge editing methods can reliably patch isolated facts, they frequently suffer from a "Reasoning Gap", where the model recalls the edited fact but fails to utilize it in multi-step reasoning chains. To bridge this gap, we introduce MCircKE (\underline{M}echanistic \underline{Circ}uit-based \underline{K}nowledge \underline{E}diting), a novel framework that enables a precise "map-and-adapt" editing procedure. MCircKE first identifies the causal circuits responsible for a specific reasoning task, capturing both the storage of the fact and the routing of its logical consequences. It then surgically update parameters exclusively within this mapped circuit. Extensive experiments on the MQuAKE-3K benchmark demonstrate the effectiveness of the proposed method for multi-hop reasoning in knowledge editing.

Related papers

• Benchmarking Knowledge Editing using Logical Rules [6.42566059684319]
  We introduce a new benchmark designed to evaluate how knowledge editing methods handle the logical consequences of a single fact edit.<n>Our findings indicate that while existing knowledge editing approaches can accurately insert direct assertions into LLMs, they frequently fail to inject entailed knowledge.<n>This highlights the critical need for semantics-aware evaluation frameworks in knowledge editing.
  arXiv  Detail & Related papers  (2026-06-09T08:21:56Z)
• Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models [12.330598426331091]
  Localized and Disentangled Knowledge Editing achieves precise and generalized editing by localizing fact-specific model layers and disentangling target-relevant inputs from irrelevant ones.<n>Experiments demonstrate LDKE achieves superior performance in propagating edits to related contexts while maintaining high locality.
  arXiv  Detail & Related papers  (2026-05-28T12:06:39Z)
• The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing [52.315369633116255]
  Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE)<n>We take a mechanistic view of KE using neuron-level knowledge attribution (NLKA)<n>Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact.<n>Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions.
  arXiv  Detail & Related papers  (2026-03-21T12:40:15Z)
• Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping [51.33479921152361]
  Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability.<n>Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors.<n>We introduce Reasoning Editing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways.
  arXiv  Detail & Related papers  (2026-03-06T22:51:39Z)
• Representation Interventions Enable Lifelong Unstructured Knowledge Control [54.86207134539453]
  Large language models (LLMs) often produce incorrect or outdated content. Updating their knowledge efficiently and accurately without costly retraining is a major challenge.<n>We introduce RILKE, a robust and scalable method that treats knowledge control as interventions within the model's representation space.<n>During training, RILKE learns paraphrase-robust and edit-localized modules that limit each update to a low-dimensional subspace to minimize cross-edit interference.<n>In inference, a query-adaptive router selects the appropriate module to guide the model's generation.
  arXiv  Detail & Related papers  (2025-11-25T22:15:00Z)
• Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition [32.73672881869734]
  We find that existing retrieval-augmented generation (RAG)-based knowledge editing methods struggle with multi-hop question answering.<n>We propose a novel Iterative Retrieval-Augmented Knowledge Editing method with guided decomposition.<n> Experimental results demonstrate that IRAKE mitigates the failure of editing caused by edit skipping and outperforms state-of-the-art methods for KE in multi-hop question answering.
  arXiv  Detail & Related papers  (2025-09-09T09:49:23Z)
• MEMOIR: Lifelong Model Editing with Minimal Overwrite and Informed Retention for LLMs [76.28901550926021]
  Existing methods for lifelong model editing compromise generalization, interfere with past edits, or fail to scale to long editing sequences.<n>We propose MEMOIR, a novel scalable framework that injects knowledge through a residual memory, while preserving the core capabilities of the pre-trained model.<n>MeMOIR achieves state-of-the-art performance across reliability, generalization, and locality metrics, scaling to thousands of sequential edits with minimal forgetting.
  arXiv  Detail & Related papers  (2025-06-09T16:16:42Z)
• Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing [12.5122702720856]
  In-context editing (ICE) offers a lightweight solution by injecting new knowledge directly into the input context.<n>Existing ICE approaches do not explicitly separate the newly injected knowledge from the model's original reasoning process.<n>We propose DecKER, a novel ICE framework that decouples reasoning from knowledge editing by generating a masked reasoning path.
  arXiv  Detail & Related papers  (2025-05-31T12:51:12Z)
• CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners [109.87058236007907]
  CaKE (Circuit-aware Knowledge Editing) is a novel method that enhances the effective integration of updated knowledge in large language models.<n> Experiments show that CaKE enables more accurate and consistent use of edited knowledge across related reasoning tasks.
  arXiv  Detail & Related papers  (2025-03-20T17:14:34Z)
• Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing [38.590823330865845]
  Large language models (LLMs) face challenges with internal knowledge inaccuracies and outdated information. Knowledge editing has emerged as a pivotal approach to mitigate these issues. We propose a novel knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE)
  arXiv  Detail & Related papers  (2024-08-22T14:53:33Z)
• WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models [78.22291694903659]
  Large language models (LLMs) need knowledge updates to meet the ever-growing world facts and correct the hallucinated responses.<n>Where the updated knowledge resides in memories is a fundamental question for model editing.<n>We propose WISE to bridge the gap between memories.
  arXiv  Detail & Related papers  (2024-05-23T16:35:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.