CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing

• URL: http://arxiv.org/abs/2603.19297v1
• Date: Wed, 11 Mar 2026 04:11:49 GMT
• Title: CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing
• Abstract summary: We introduce CLaRE, a representation-level technique to identify where ripple effects may occur.<n>CLaRE quantifies entanglement between facts using forward activations from a single intermediate layer, avoiding costly backward passes.<n>We compute large-scale entanglement graphs of this corpus for multiple models, capturing how local edits propagate through representational space.
• Score: 4.180400747723904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The static knowledge representations of large language models (LLMs) inevitably become outdated or incorrect over time. While model-editing techniques offer a promising solution by modifying a model's factual associations, they often produce unpredictable ripple effects, which are unintended behavioral changes that propagate even to the hidden space. In this work, we introduce CLaRE, a lightweight representation-level technique to identify where these ripple effects may occur. Unlike prior gradient-based methods, CLaRE quantifies entanglement between facts using forward activations from a single intermediate layer, avoiding costly backward passes. To enable systematic study, we prepare and analyse a corpus of 11,427 facts drawn from three existing datasets. Using CLaRE, we compute large-scale entanglement graphs of this corpus for multiple models, capturing how local edits propagate through representational space. These graphs enable stronger preservation sets for model editing, audit trails, efficient red-teaming, and scalable post-edit evaluation. In comparison to baselines, CLaRE achieves an average of 62.2% improvement in Spearman correlation with ripple effects while being $2.74\times$ faster, and using $2.85\times$ less peak GPU memory. Besides, CLaRE requires only a fraction of the storage needed by the baselines to compute and preserve fact representations. Our entanglement graphs and corpus are available at https://anonymous.4open.science/r/CLaRE-488E.

Related papers

• Low-Rank Ternary Adaptation for Fine-Tuning Transformers [54.3230438745856]
  Ternary transformers offer extreme memory and compute efficiency.<n>Existing low-bit LoRA-based methods cannot directly fine-tune ternary weights.<n>We propose ternary multiplicative adaptation, which represents discrete updates of ternary weights.
  arXiv  Detail & Related papers  (2026-08-25T12:15:56Z)
• Scaling Laws for Behavioral Foundation Models over User Event Sequences [2.924581427482972]
  We study a common two-part behavioral-model architecture, a feature-based event embedder, and a decoder-only transformer.<n>Across roughly 600 runs on real interaction data, spanning $1015$-$1019$ training FLOPs, we vary four deployment-relevant axes.<n> Compute-optimal training is data-heavy relative to text at low compute, but its $D/N$ ratio moves toward the Chinchilla as compute increases.
  arXiv  Detail & Related papers  (2026-06-03T15:59:25Z)
• LEADER: Learning Reliable Local-to-Global Correspondences for LiDAR Relocalization [44.37632368250295]
  We propose LEADER, a robust LiDAR-based relocalization framework enhanced by a simple, yet effective geometric encoder.<n>Experiments on the Oxford RobotCar and NCLT datasets demonstrate that LEADER outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-13T11:52:29Z)
• Spatial Autoregressive Modeling of DINOv3 Embeddings for Unsupervised Anomaly Detection [15.896078006029475]
  DINO models provide rich patch-level representations that have recently enabled strong performance in unsupervised anomaly detection (UAD)<n>Most existing methods extract patch embeddings from normal'' images and model them independently, ignoring spatial and neighborhood relationships between patches.<n>We propose a framework that explicitly models spatial and contextual dependencies between patch embeddings using a 2D autoregressive (AR) model.
  arXiv  Detail & Related papers  (2026-03-03T13:30:33Z)
• LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution [62.830878652285406]
  Training data attribution identifies which training examples most influenced a model's prediction.<n>LoRIF exploits low-rank structures of gradient to address both bottlenecks.<n>On models from 0.1B to 70B parameters trained on datasets with millions of examples, LoRIF achieves up to 20$times$ storage reduction and query-time speedup.
  arXiv  Detail & Related papers  (2026-01-29T16:18:34Z)
• Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints [14.20716202034732]
  Full fine-tuning of Large Language Models (LLMs) is notoriously memory-intensive.<n>We introduce GradLite, a backward-friendly solution that relaxes the requirement of exact gradients.<n>We show that GradLite maintains unbiased estimates with bounded variance, ensuring convergence rates comparable to Adam.
  arXiv  Detail & Related papers  (2025-10-26T00:50:12Z)
• Rethinking the Residual Distribution of Locate-then-Editing Methods in Model Editing [14.958557185068]
  Model editing enables targeted updates to the knowledge of large language models.<n> locate-then-edit methods first identify critical layers, then compute residuals at the final critical layer based on the target edit.<n> residual distribution, a core mechanism in these methods, introduces weight shift errors that undermine editing precision.<n>We propose the BLUE strategy to enhance locate-then-edit methods.
  arXiv  Detail & Related papers  (2025-02-06T03:20:17Z)
• LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributions [17.88069510398486]
  gradient-based explanations struggle with Transformers, and how can we improve them? We identify flow imbalances in Transformers that violate FullGrad-completeness, a critical property for attribution gradient that CNNs naturally possess. We introduce LibraGrad -- a theoretically grounded post-hoc approach that corrects gradient imbalances through pruning and scaling of backward paths.
  arXiv  Detail & Related papers  (2024-11-24T15:02:52Z)
• Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models [56.00251589760559]
  Large language models (LLMs) can act as gradient priors in a zero-shot setting.<n>We introduce LM-GC, a novel method that integrates LLMs with arithmetic coding.<n>Experiments indicate that LM-GC surpasses existing state-of-the-art lossless compression methods.
  arXiv  Detail & Related papers  (2024-09-26T13:38:33Z)
• Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness [86.61582747039053]
  Language model training in distributed settings is limited by the communication cost of exchanges. We extend recent work using shared randomness to perform distributed fine-tuning with low bandwidth.
  arXiv  Detail & Related papers  (2023-06-16T17:59:51Z)
• Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model [89.8764435351222]
  We propose a new family of unbiased estimators called WTA-CRS, for matrix production with reduced variance. Our work provides both theoretical and experimental evidence that, in the context of tuning transformers, our proposed estimators exhibit lower variance compared to existing ones.
  arXiv  Detail & Related papers  (2023-05-24T15:52:08Z)
• Learning Decorrelated Representations Efficiently Using Fast Fourier Transform [3.932322649674071]
  We propose a relaxed decorrelating regularizer that can be computed in O(n d log d) time by Fast Fourier Transform. The proposed regularizer exhibits accuracy comparable to that of existing regularizers in downstream tasks, whereas their training requires less memory and is faster for large d.
  arXiv  Detail & Related papers  (2023-01-04T12:38:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.