BetaEdit: Null-Space Constrained Sequential Model Editing

• URL: http://arxiv.org/abs/2605.09285v1
• Date: Sun, 10 May 2026 03:17:57 GMT
• Title: BetaEdit: Null-Space Constrained Sequential Model Editing
• Abstract summary: Null-space-based methods rely on an approximate null space--leading to knowledge leakage.<n>Recent work shows that history-aware editing strategies can mitigate this decline, yet the underlying reason remains unclear.<n>We propose BetaEdit, a framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm.
• Score: 7.908184111860611
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.

Related papers

• Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models" [0.0]
  We present a study of AlphaEdit, reproducing its reported results under the original experimental setup.<n>We extend evaluation of edited models on three extra benchmarks, namely, BoolQ, HellaSwag, and XSTest.<n>Our results confirm that AlphaEdit performs as reported within its original scope.
  arXiv  Detail & Related papers  (2026-06-25T09:20:00Z)
• The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models [15.376868559569623]
  We first analyze the empirical success of AlphaEdit and establish, via a rigorous optimization analysis, the formal equivalence between one-time and sequential editing.<n>We empirically confirm that many commonly used regularization strategies are unnecessary for reliable sequential updates.<n>Our work provides Ariadne's thread through the labyrinth of sequential editing, charting a path toward simpler, more interpretable, and dependable knowledge updates.
  arXiv  Detail & Related papers  (2026-05-26T08:07:25Z)
• EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing [19.834477925624658]
  Large language models (LLMs) require continual updates to rectify outdated or erroneous knowledge.<n>Existing approaches are mainly based on a locate-then-edit framework.<n>We introduce EvoEdit, a novel editing strategy that mitigates catastrophic interference through sequential null-space alignment.
  arXiv  Detail & Related papers  (2025-10-11T21:36:14Z)
• Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation [50.40861036534546]
  Large language models (LLMs) inevitably encode outdated or incorrect knowledge. Updating, deleting, and forgetting such knowledge is important for alignment, safety, and other issues.<n>To address this issue, model editing has emerged as a promising paradigm: by precisely editing a small subset of parameters such that a specific fact is updated while preserving other knowledge.<n>Despite its great success reported in previous papers, we find the apparent reliability of editing rests on a fragile foundation.<n>Our empirical evidence shows that editing is likely to be based on shortcuts rather than full semantics, calling for an urgent reconsideration of the very basis of model editing before further advancements can
  arXiv  Detail & Related papers  (2025-10-01T07:59:23Z)
• Model Merging for Knowledge Editing [53.799891745131724]
  Large Language Models (LLMs) require continuous updates to maintain accurate and current knowledge as the world evolves.<n>Existing knowledge editing approaches offer various solutions for knowledge updating, but they often struggle with sequential editing scenarios.<n>This paper proposes a two-stage framework combining robust supervised fine-tuning (R-SFT) with model merging for knowledge editing.
  arXiv  Detail & Related papers  (2025-06-14T07:42:39Z)
• LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editing [28.870053452479443]
  Current locate-then-edit approaches exhibit a progressive performance decline during sequential editing.<n>textbfLyapLock is proposed to decompose the long-term constrained programming into tractable stepwise subproblems for efficient solving.<n> Experimental results show that our framework scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89% over SOTA baselines.
  arXiv  Detail & Related papers  (2025-05-21T16:16:33Z)
• AnyEdit: Edit Any Knowledge Encoded in Language Models [76.28789588247659]
  We propose AnyEdit, a new autoregressive editing paradigm for large language models (LLMs)<n>It decomposes long-form knowledge into sequential chunks and iteratively edits the key token in each chunk, ensuring consistent and accurate outputs.<n>It outperforms strong baselines by 21.5% on benchmarks including UnKEBench, AKEW, and our new EditEverything dataset for long-form diverse-formatted knowledge.
  arXiv  Detail & Related papers  (2025-02-08T16:18:37Z)
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models [63.209935157623946]
  Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge.<n>We introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters.<n>We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge.
  arXiv  Detail & Related papers  (2024-10-03T10:06:27Z)
• Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge [93.54427119091174]
  We propose outDated ISsue aware deCOding to enhance the performance of edited models on reasoning questions. We capture the difference in the probability distribution between the original and edited models. We amplify the difference of the token prediction in the edited model to alleviate the outdated issue.
  arXiv  Detail & Related papers  (2024-06-05T03:00:15Z)
• Memory-Based Model Editing at Scale [102.28475739907498]
  Existing model editors struggle to accurately model an edit's intended scope. We propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC) SERAC stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed.
  arXiv  Detail & Related papers  (2022-06-13T23:40:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.