Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing

• URL: http://arxiv.org/abs/2506.00536v1
• Date: Sat, 31 May 2025 12:51:12 GMT
• Title: Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing
• Authors: Changyue Wang, Weihang Su, Qingyao Ai, Yujia Zhou, Yiqun Liu,
• Abstract summary: In-context editing (ICE) offers a lightweight solution by injecting new knowledge directly into the input context.<n>Existing ICE approaches do not explicitly separate the newly injected knowledge from the model's original reasoning process.<n>We propose DecKER, a novel ICE framework that decouples reasoning from knowledge editing by generating a masked reasoning path.
• Score: 12.5122702720856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge editing aims to efficiently update Large Language Models (LLMs) by modifying specific knowledge without retraining the entire model. Among knowledge editing approaches, in-context editing (ICE) offers a lightweight solution by injecting new knowledge directly into the input context, leaving model parameters unchanged. However, existing ICE approaches do not explicitly separate the newly injected knowledge from the model's original reasoning process. This entanglement often results in conflicts between external updates and internal parametric knowledge, undermining the consistency and accuracy of the reasoning path.In this work, we conduct preliminary experiments to examine how parametric knowledge influences reasoning path planning. We find that the model's reasoning is tightly coupled with its internal knowledge, and that naively injecting new information without adapting the reasoning path often leads to performance degradation, particularly in multi-hop tasks. To this end, we propose DecKER, a novel ICE framework that decouples reasoning from knowledge editing by generating a masked reasoning path and then resolving knowledge edits via hybrid retrieval and model-based validation. Experiments on multi-hop QA benchmarks show that DecKER significantly outperforms existing ICE methods by mitigating knowledge conflicts and preserving reasoning consistency. Our code is available at: https://github.com/bebr2/DecKER .

Related papers

• Model Merging for Knowledge Editing [53.799891745131724]
  Large Language Models (LLMs) require continuous updates to maintain accurate and current knowledge as the world evolves.<n>Existing knowledge editing approaches offer various solutions for knowledge updating, but they often struggle with sequential editing scenarios.<n>This paper proposes a two-stage framework combining robust supervised fine-tuning (R-SFT) with model merging for knowledge editing.
  arXiv  Detail & Related papers  (2025-06-14T07:42:39Z)
• PropMEND: Hypernetworks for Knowledge Propagation in LLMs [82.99849359892112]
  We present a hypernetwork-based approach for knowledge propagation, named PropMEND.<n>We show almost 2x accuracy on challenging multi-hop questions whose answers are not explicitly stated in the injected fact.<n>We also introduce a new dataset, Controlled RippleEdit, to evaluate the generalization of our hypernetwork.
  arXiv  Detail & Related papers  (2025-06-10T15:44:19Z)
• CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners [88.35958039968081]
  CaKE (Circuit-aware Knowledge Editing) is a novel method that enables more effective integration of updated knowledge in large language models.<n>Results show that CaKE enables more accurate and consistent use of updated knowledge across related reasoning tasks.
  arXiv  Detail & Related papers  (2025-03-20T17:14:34Z)
• Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning [38.018263569983226]
  We provide an evaluation of ten model editing methods along four dimensions: reliability, generalization, locality, and portability.<n>We then propose a straightforward method called Selective Contextual Reasoning (SCR) for knowledge updating.
  arXiv  Detail & Related papers  (2025-03-07T08:04:25Z)
• CoME: An Unlearning-based Approach to Conflict-free Model Editing [8.215201299292033]
  Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability.<n>We propose Conflict-free Model Editing (CoME), a novel framework that enhances the accuracy of knowledge updates in LLMs by selectively removing outdated knowledge.
  arXiv  Detail & Related papers  (2025-02-20T04:55:38Z)
• Related Knowledge Perturbation Matters: Rethinking Multiple Pieces of Knowledge Editing in Same-Subject [49.559994791305535]
  Current state-of-the-art editing methods struggle when tasked with editing multiple related knowledge pieces for the same subject.<n>We introduce the $textS2textRKE$(Same-Subject Related Knowledge Editing) benchmark.<n>Our experiments reveal that only mainstream locate-then-edit methods, such as ROME and MEMIT, exhibit "related knowledge perturbation"
  arXiv  Detail & Related papers  (2025-02-08T04:47:17Z)
• Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing [38.590823330865845]
  Large language models (LLMs) face challenges with internal knowledge inaccuracies and outdated information. Knowledge editing has emerged as a pivotal approach to mitigate these issues. We propose a novel knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE)
  arXiv  Detail & Related papers  (2024-08-22T14:53:33Z)
• Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge [93.54427119091174]
  We propose outDated ISsue aware deCOding to enhance the performance of edited models on reasoning questions. We capture the difference in the probability distribution between the original and edited models. We amplify the difference of the token prediction in the edited model to alleviate the outdated issue.
  arXiv  Detail & Related papers  (2024-06-05T03:00:15Z)
• Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks [36.292901021210575]
  We introduce a novel reasoning-based benchmark -- ReCoE (Reasoning-based Counterfactual Editing dataset) We conduct a thorough analysis of existing knowledge editing techniques, including input augmentation, finetuning, and locate-and-edit. All model editing methods show notably low performance on this dataset, especially in certain reasoning schemes.
  arXiv  Detail & Related papers  (2024-01-31T04:12:59Z)
• A Comprehensive Study of Knowledge Editing for Large Language Models [82.65729336401027]
  Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. This paper defines the knowledge editing problem and provides a comprehensive review of cutting-edge approaches. We introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches.
  arXiv  Detail & Related papers  (2024-01-02T16:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.