CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing

• URL: http://arxiv.org/abs/2602.15823v1
• Date: Tue, 17 Feb 2026 18:58:04 GMT
• Title: CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
• Authors: Zarif Ikram, Arad Firouzkouhi, Stephen Tu, Mahdi Soltanolkotabi, Paria Rashidinejad,
• Abstract summary: CrispEdit is a principled second-order editing algorithm that treats capability preservation as an explicit constraint.<n>We show that CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets.
• Score: 25.410131838577907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.

Related papers

• Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing [16.768713543557443]
  Model editing updates a pre-trained with new facts or rules without re-training, while preserving unrelated behavior.<n>We propose RLSEdit, a least-squares editor for long sequential editing.<n>Experiments on multiple model families demonstrate stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability.
  arXiv  Detail & Related papers  (2026-01-22T06:11:44Z)
• Fine-tuning Done Right in Model Editing [83.79661791576103]
  Fine-tuning, a foundational method for adapting large language models, has long been considered ineffective for model editing.<n>We restore fine-tuning to the standard breadth-first (i.e., epoch-based) pipeline with mini-batch optimization.<n>We derive LocFT-BF, a simple and effective localized editing method built on the restored fine-tuning framework.
  arXiv  Detail & Related papers  (2025-09-26T08:53:13Z)
• Visual Autoregressive Modeling for Instruction-Guided Image Editing [97.04821896251681]
  We present a visual autoregressive framework that reframes image editing as a next-scale prediction problem.<n>VarEdit generates multi-scale target features to achieve precise edits.<n>It completes a $512times512$ editing in 1.2 seconds, making it 2.2$times$ faster than the similarly sized UltraEdit.
  arXiv  Detail & Related papers  (2025-08-21T17:59:32Z)
• InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing [86.17245523439514]
  In-context learning is a promising editing method by comprehending edit information through context encoding.<n>This method is constrained by the limited context window of large language models.<n>We propose InComeS, a flexible framework that enhances LLMs' ability to process editing contexts.
  arXiv  Detail & Related papers  (2025-05-28T09:20:18Z)
• LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editing [28.870053452479443]
  Current locate-then-edit approaches exhibit a progressive performance decline during sequential editing.<n>textbfLyapLock is proposed to decompose the long-term constrained programming into tractable stepwise subproblems for efficient solving.<n> Experimental results show that our framework scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89% over SOTA baselines.
  arXiv  Detail & Related papers  (2025-05-21T16:16:33Z)
• Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model [60.82962950960996]
  We introduce UnifyEdit, a tuning-free method that performs diffusion latent optimization.<n>We develop two attention-based constraints: a self-attention (SA) preservation constraint for structural fidelity, and a cross-attention (CA) alignment constraint to enhance text alignment.<n>Our approach achieves a robust balance between structure preservation and text alignment across various editing tasks, outperforming other state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-08T01:02:50Z)
• Constraining Sequential Model Editing with Editing Anchor Compression [40.93064933191375]
  Large language models (LLMs) struggle with hallucinations due to false or outdated knowledge.<n>This paper statistically observes that the parameter matrix after editing exhibits a significant deviation compared to its previous state as the number of edits increases.<n>A framework termed Editing Anchor Compression (EAC) is proposed to constrain the deviation of the parameter matrix during sequential editing.
  arXiv  Detail & Related papers  (2025-02-25T03:56:49Z)
• Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing [60.730661748555214]
  We introduce textbfTask-textbfOriented textbfDiffusion textbfInversion (textbfTODInv), a novel framework that inverts and edits real images tailored to specific editing tasks. ToDInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability.
  arXiv  Detail & Related papers  (2024-08-23T22:16:34Z)
• Perturbation-Restrained Sequential Model Editing [33.51709226068619]
  Current model editing methods compromise the general abilities of large language models (LLMs) as the number of edits increases.<n>A framework termed Perturbation Restraint on Upper bouNd for Editing (PRUNE) is proposed, which applies the condition number restraints in sequential editing.<n>The results show that PRUNE can preserve general abilities while maintaining the editing performance effectively in sequential model editing.
  arXiv  Detail & Related papers  (2024-05-27T04:40:56Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• A Structural Model for Contextual Code Changes [20.185486717922615]
  Given a code snippet that is partially edited, our goal is to predict a completion of the edit for the rest of the snippet. Our model achieves a 28% relative gain over state-of-the-art sequential models and 2x higher accuracy than syntactic models that learn to generate the edited code.
  arXiv  Detail & Related papers  (2020-05-27T07:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.