Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression

• URL: http://arxiv.org/abs/2604.19089v1
• Date: Tue, 21 Apr 2026 05:02:29 GMT
• Title: Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression
• Abstract summary: Large language models (LLMs) require frequent knowledge updates to reflect changing facts and hallucinations.<n>Existing parameter editing methods struggle with stability during sequential edits due to catastrophic forgetting.<n>Our framework first selects relevant knowledge from retrieved information to modify the query effectively.<n>It then incorporates a decoding strategy to suppress the model's original knowledge probabilities.
• Score: 23.273954918352903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) require frequent knowledge updates to reflect changing facts and mitigate hallucinations. To meet this demand, lifelong knowledge editing has emerged as a continual approach to modify specific pieces of knowledge without retraining the entire model. Existing parameter editing methods struggle with stability during sequential edits due to catastrophic forgetting. While retrieval-based approaches are proposed to alleviate this issue, their applicability remains limited across various datasets because of high training costs. To address these limitations and enhance scalability in lifelong settings, we propose LightEdit. Our framework first selects relevant knowledge from retrieved information to modify the query effectively. It then incorporates a decoding strategy to suppress the model's original knowledge probabilities, thereby enabling efficient edits based on the selected information. Extensive experiments on ZSRE, Counterfact, and RIPE benchmarks demonstrate that LightEdit outperforms existing lifelong knowledge editing methods. Furthermore, by minimizing training costs, LightEdit achieves cost-effective scalability, enabling easy adaptation to various datasets.

Related papers

• EvoEdit: Lifelong Free-Text Knowledge Editing through Latent Perturbation Augmentation and Knowledge-driven Parameter Fusion [31.09201415423854]
  We propose Lifelong Free-text Knowledge Editing (LF-Edit)<n>It enables models to incorporate updates expressed in natural language and supports continual editing over time.<n>Despite its promise, LF-Edit faces the dual challenge of integrating new knowledge while mitigating the forgetting of prior information.
  arXiv  Detail & Related papers  (2025-12-04T07:55:36Z)
• Representation Interventions Enable Lifelong Unstructured Knowledge Control [54.86207134539453]
  Large language models (LLMs) often produce incorrect or outdated content. Updating their knowledge efficiently and accurately without costly retraining is a major challenge.<n>We introduce RILKE, a robust and scalable method that treats knowledge control as interventions within the model's representation space.<n>During training, RILKE learns paraphrase-robust and edit-localized modules that limit each update to a low-dimensional subspace to minimize cross-edit interference.<n>In inference, a query-adaptive router selects the appropriate module to guide the model's generation.
  arXiv  Detail & Related papers  (2025-11-25T22:15:00Z)
• Retention analysis of edited knowledge after fine-tuning [5.1877231178075425]
  Large language models (LLMs) store vast amounts of knowledge, which often requires updates to correct factual errors, incorporate newly acquired information, or adapt model behavior.<n>Model editing methods have emerged as efficient solutions for such updates, offering localized and precise knowledge modification at significantly lower computational cost than continual training.<n>However, the effect of fine-tuning on previously edited knowledge remains poorly understood.
  arXiv  Detail & Related papers  (2025-07-14T15:51:19Z)
• ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphs [3.9295613363026174]
  We present ThinkEval, a framework to quantify indirect knowledge leakage and ripple effects in model-editing.<n>ThinkEval builds and employs specialized knowledge graphs to analyze the causal structure of facts before and after editing.<n>We evaluate five editing techniques: AlphaEdit, RECT, ROME, MEMIT, and PRUNE.
  arXiv  Detail & Related papers  (2025-06-02T07:24:12Z)
• InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing [86.17245523439514]
  In-context learning is a promising editing method by comprehending edit information through context encoding.<n>This method is constrained by the limited context window of large language models.<n>We propose InComeS, a flexible framework that enhances LLMs' ability to process editing contexts.
  arXiv  Detail & Related papers  (2025-05-28T09:20:18Z)
• WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs [66.51203413345773]
  We bridge research into lifelong knowledge editing to real-world edits at a practically relevant scale.<n>We first introduce WikiBigEdit; a large-scale benchmark of real-world Wikidata edits.<n>In its first instance, it includes over 500K question-answer pairs for knowledge editing.
  arXiv  Detail & Related papers  (2025-03-07T18:45:42Z)
• K-Edit: Language Model Editing with Contextual Knowledge Awareness [71.73747181407323]
  Knowledge-based model editing enables precise modifications to the weights of large language models.<n>We present K-Edit, an effective approach to generating contextually consistent knowledge edits.
  arXiv  Detail & Related papers  (2025-02-15T01:35:13Z)
• Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning [30.554641380670315]
  We introduce RECIPE, a ContInuous Prompt lEarning method to boost editing efficacy and inference efficiency in lifelong learning.<n> RECIPE first converts knowledge statements into short and informative continuous prompts, prefixed to the LLM's input query embedding.<n>It further integrates the Knowledge Sentinel (KS) that acts as an intermediary to calculate a dynamic threshold.<n>Our retriever and prompt encoder are jointly trained to achieve editing properties, i.e. reliability, generality, and locality.
  arXiv  Detail & Related papers  (2024-05-06T08:52:11Z)
• EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries [69.72012539060731]
  We introduce a theoretical framework for efficient knowledge editing (KE) in large language models (LLMs) We propose a novel task of event-based knowledge editing that pairs facts with event descriptions. We empirically demonstrate the superiority of event-based editing over the existing setting on resolving uncertainty in edited models.
  arXiv  Detail & Related papers  (2024-02-17T16:34:50Z)
• SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering [17.20346072074533]
  Recent model editing is a promising technique for efficiently updating a small amount of knowledge of large language models.<n>We propose a detachable and expandable Subject Word Embedding Altering (SWEA) framework, which finds the editing embeddings through token-level matching.<n>We demonstrate the overall state-of-the-art (SOTA) performance of SWEA$oplus$OS on the CounterFact and zsRE datasets.
  arXiv  Detail & Related papers  (2024-01-31T13:08:45Z)
• A Comprehensive Study of Knowledge Editing for Large Language Models [82.65729336401027]
  Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. This paper defines the knowledge editing problem and provides a comprehensive review of cutting-edge approaches. We introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches.
  arXiv  Detail & Related papers  (2024-01-02T16:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.