MetaKE: Meta-learning Aligned Knowledge Editing via Bi-level Optimization

• URL: http://arxiv.org/abs/2603.12677v2
• Date: Mon, 16 Mar 2026 01:43:55 GMT
• Title: MetaKE: Meta-learning Aligned Knowledge Editing via Bi-level Optimization
• Abstract summary: State-of-the-art methods suffer from an open-loop control mismatch.<n>We identify a critical "Semantic-Execution Disconnect"<n>We propose MetaKE, a new framework that reframes KE as a bi-level optimization problem.
• Score: 11.430206422495829
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge editing (KE) aims to precisely rectify specific knowledge in Large Language Models (LLMs) without disrupting general capabilities. State-of-the-art methods suffer from an open-loop control mismatch. We identify a critical "Semantic-Execution Disconnect": the semantic target is derived independently without feedback from the downstream's feasible region. This misalignment often causes valid semantic targets to fall within the prohibited space, resulting in gradient truncation and editing failure. To bridge this gap, we propose MetaKE (Meta-learning Aligned Knowledge Editing), a new framework that reframes KE as a bi-level optimization problem. Departing from static calculation, MetaKE treats the edit target as a learnable meta-parameter: the upper-level optimizer seeks a feasible target to maximize post-edit performance, while the lower-level solver executes the editing. To address the challenge of differentiating through complex solvers, we derive a Structural Gradient Proxy, which explicitly backpropagates editability constraints to the target learning phase. Theoretical analysis demonstrates that MetaKE automatically aligns the edit direction with the model's feasible manifold. Extensive experiments confirm that MetaKE significantly outperforms strong baselines, offering a new perspective on knowledge editing.

Related papers

• The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing [52.315369633116255]
  Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE)<n>We take a mechanistic view of KE using neuron-level knowledge attribution (NLKA)<n>Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact.<n>Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions.
  arXiv  Detail & Related papers  (2026-03-21T12:40:15Z)
• Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning [46.514554089834554]
  Existing editing methods rely on a rigid mapping from parameter or module modifications to output.<n>In this paper, we reformulate MLLM editing as an out-of-distribution (OOD) generalization problem.<n>We propose ODEdit, a plug-and-play invariant learning based framework that enhances editing reliability, locality, and generality.
  arXiv  Detail & Related papers  (2026-01-27T15:25:07Z)
• Are We Evaluating the Edit Locality of LLM Model Editing Properly? [68.441768731381]
  We find that existing specificity evaluation protocols are inadequate for this purpose.<n>Existing specificity metrics are weakly correlated with the strength of specificity regularizers.<n>We also find that current metrics lack sufficient sensitivity, rendering them ineffective at distinguishing the specificity performance of different methods.
  arXiv  Detail & Related papers  (2026-01-24T07:07:21Z)
• Consistency-Aware Editing for Entity-level Unlearning in Language Models [53.522931419965424]
  We introduce a novel consistency-aware editing (CAE) framework for entity-level unlearning.<n>CAE aggregates a diverse set of prompts related to a target entity, including its attributes, relations, and adversarial paraphrases.<n>It then jointly learns a low-rank update guided by a consistency regularizer that aligns the editing directions across prompts.
  arXiv  Detail & Related papers  (2025-12-19T15:18:07Z)
• An Information-Theoretic Framework for Robust Large Language Model Editing [17.984683741974063]
  Large Language Models (LLMs) have become indispensable tools in science, technology, and society.<n>Errors or outdated information within these models can undermine their accuracy and restrict their safe deployment.<n>We introduce a novel framework for editing LLMs, grounded in information bottleneck theory.<n>We present the Information Bottleneck Knowledge Editor (IBKE), which leverages compact latent representations to guide gradient-based updates.
  arXiv  Detail & Related papers  (2025-12-18T06:21:17Z)
• EtCon: Edit-then-Consolidate for Reliable Knowledge Editing [85.20993502078899]
  We propose Edit-then-Consolidate, a novel knowledge editing paradigm that aims to bridge the gap between theoretical knowledge editing methods and their real-world applicability.<n>Our framework consistently improves editing reliability and generalization under real-world evaluations, while better preserving locality and pre-trained capabilities.
  arXiv  Detail & Related papers  (2025-12-04T12:43:50Z)
• Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs [71.8547241246169]
  We introduce CogEdit, a novel benchmark designed to evaluate MLLMs' meta-cognitive knowledge editing abilities.<n>We propose MIND, a framework that constructs a meta-knowledge memory for self-awareness, employs game-theoretic interactions to monitor knowledge activation, and incorporates label refinement for noise-robust updates.
  arXiv  Detail & Related papers  (2025-09-06T13:26:04Z)
• REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing [42.89229070245538]
  We introduce REACT, a framework for precise and controllable knowledge editing.<n>In the initial phase, we utilize tailored stimuli to extract latent factual representations.<n>In the second phase, we apply controllable perturbations to hidden states using the obtained vector with a magnitude scalar.
  arXiv  Detail & Related papers  (2025-05-25T01:57:06Z)
• Uncovering Overfitting in Large Language Model Editing [35.55260822503773]
  We identify and investigate the phenomenon of Editing Overfit, where edited models assign disproportionately high probabilities to the edit target.<n>We propose Learn the Inference (LTI), which introduce a Multi-stage Inference Constraint module to guide the edited models in recalling new knowledge.
  arXiv  Detail & Related papers  (2024-10-10T11:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.