HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning

• URL: http://arxiv.org/abs/2604.11214v1
• Date: Mon, 13 Apr 2026 09:08:57 GMT
• Title: HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning
• Abstract summary: HiEdit is a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance.<n> Experiments show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit.
• Score: 16.114622048845895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances. This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model. Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge. To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance. By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates. Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit. Our code is available at: https://github.com/yangfanww/hiedit.

Related papers

• Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models [56.355176278601476]
  Large language models store not only isolated facts but also rules that support reasoning across symbolic expressions, natural language explanations, and concrete instances.<n>Most model editing methods are built for fact-level knowledge, assuming that a target edit can be achieved through a localized intervention.<n>We investigate this problem through a mechanistic study of rule-level knowledge editing.
  arXiv  Detail & Related papers  (2026-04-09T14:22:19Z)
• Quantifying Edits Decay in Fine-tuned LLMs [17.377278510871843]
  This study investigates how fine-tuning affects knowledge editing.<n>We evaluate two state-of-the-art editing methods (MEMIT, AlphaEdit) and three fine-tuning approaches.<n>Our results show that edits decay after fine-tuning, with survival varying across configurations.
  arXiv  Detail & Related papers  (2025-11-08T04:58:03Z)
• Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model Editing [10.54738347540608]
  Large Language Models (LLMs) are widely deployed in downstream tasks, but keeping their knowledge up-to-date via retraining or fine-tuning is often computationally expensive.<n>Model editing provides a more efficient alternative by updating a targeted subset of parameters, which often follows the locate-and-edit paradigm.<n>We propose two complementary methods: iterative model editing, which applies successive edits to mitigate UnderEdit, and neighbor-assisted model editing, which incorporates neighboring knowledge during editing to reduce OverEdit.
  arXiv  Detail & Related papers  (2025-03-14T21:53:12Z)
• Constraining Sequential Model Editing with Editing Anchor Compression [40.93064933191375]
  Large language models (LLMs) struggle with hallucinations due to false or outdated knowledge.<n>This paper statistically observes that the parameter matrix after editing exhibits a significant deviation compared to its previous state as the number of edits increases.<n>A framework termed Editing Anchor Compression (EAC) is proposed to constrain the deviation of the parameter matrix during sequential editing.
  arXiv  Detail & Related papers  (2025-02-25T03:56:49Z)
• AnyEdit: Edit Any Knowledge Encoded in Language Models [76.28789588247659]
  We propose AnyEdit, a new autoregressive editing paradigm for large language models (LLMs)<n>It decomposes long-form knowledge into sequential chunks and iteratively edits the key token in each chunk, ensuring consistent and accurate outputs.<n>It outperforms strong baselines by 21.5% on benchmarks including UnKEBench, AKEW, and our new EditEverything dataset for long-form diverse-formatted knowledge.
  arXiv  Detail & Related papers  (2025-02-08T16:18:37Z)
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models [63.209935157623946]
  Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge.<n>We introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters.<n>We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge.
  arXiv  Detail & Related papers  (2024-10-03T10:06:27Z)
• ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA [55.697627106315004]
  Large language models (LLMs) require model editing to efficiently update specific knowledge within them and avoid factual errors.<n>Previous approaches manage sequential edits by freezing original parameters and discretely allocating new parameters for each knowledge update.<n>We propose ELDER, a novel approach to create a continuous association between data and adapters.
  arXiv  Detail & Related papers  (2024-08-19T02:27:00Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models [68.03946716358335]
  We find that we can change how a fact is stored in a model by editing weights that are in a different location than where existing methods suggest that the fact is stored. This is surprising because we would expect that localizing facts to specific model parameters would tell us where to manipulate knowledge in models. Our results suggest, counterintuitively, that better mechanistic understanding of how pretrained language models work may not always translate to insights about how to best change their behavior.
  arXiv  Detail & Related papers  (2023-01-10T21:26:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.