EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts

• URL: http://arxiv.org/abs/2605.02083v2
• Date: Tue, 05 May 2026 02:56:57 GMT
• Title: EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts
• Abstract summary: Local factual edits in scientific manuscripts often create non-local revision obligations.<n>In an audit of recent arXiv cs.CL benchmark and dataset papers, we find fact-dependent qualitative claims in 37.2% of papers.<n>We introduce EditPropBench, a benchmark for measuring whether LLM editors propagate factual edits through dependent manuscript claims.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Local factual edits in scientific manuscripts often create non-local revision obligations. If a dataset changes from 215 to 80 documents, claims such as 'medium-scale' or 'a few hundred items' may also become stale, even though they do not repeat the edited number. In an audit of recent arXiv cs.CL benchmark and dataset papers, we find fact-dependent qualitative claims in 37.2% of papers, suggesting that this dependency pattern is common in the target genre. We introduce EditPropBench, a benchmark for measuring whether LLM editors propagate factual edits through dependent manuscript claims. Each item contains an ML/NLP-style synthetic manuscript, a targeted edit, and a controlled fact graph with sentence-level labels for direct targets, required downstream updates, and unrelated text that should remain unchanged. We summarize cascade success with Edit-Ripple Adherence (ERA), the fraction of required downstream updates correctly revised, and validate the metric with adversarial probes and stress-test variants. On the hardest cases, where dependent claims use implicit or free-form wording rather than repeating the edited value, five LLM editing systems span ERA 0.148-0.705. Even the strongest misses roughly 30% of required cascade updates. This advantage persists in a mixed evaluation that includes easy cases solvable by deterministic substitution. EditPropBench shows that current LLM editors can repair many implicit consequences of factual edits, but reliable scientific revision still requires cascade-aware checking.

Related papers

• Selective Knowledge Edit Reversal via Gated Singular Vector Shrinkage [29.783345718823487]
  We study selective reversal of edited knowledge, where the goal is to reverse targeted edited facts while preserving the remaining edited facts.<n>Based on the hypothesis that each edit is sparsely encoded within the dominant subspace of the edited matrix, we propose a spectral-based reversal framework.<n> Experiments across multiple settings demonstrate the effectiveness of our method in reversing selected edits while preserving unrelated edited facts.
  arXiv  Detail & Related papers  (2026-09-02T04:32:07Z)
• MEMOIR: Lifelong Model Editing with Minimal Overwrite and Informed Retention for LLMs [76.28901550926021]
  Existing methods for lifelong model editing compromise generalization, interfere with past edits, or fail to scale to long editing sequences.<n>We propose MEMOIR, a novel scalable framework that injects knowledge through a residual memory, while preserving the core capabilities of the pre-trained model.<n>MeMOIR achieves state-of-the-art performance across reliability, generalization, and locality metrics, scaling to thousands of sequential edits with minimal forgetting.
  arXiv  Detail & Related papers  (2025-06-09T16:16:42Z)
• Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications [4.751608548909266]
  FineEdit is a specialized editing model explicitly trained for context-aware text modifications.<n>FineEdit outperforms state-of-the-art models on single-turn edits, up to 30% over Llama-3.2-3B, and exceeding Mistral-7B-OpenOrca performance by over 40% on direct editing tasks.
  arXiv  Detail & Related papers  (2025-02-19T01:41:44Z)
• The Mirage of Model Editing: Revisiting Evaluation in the Wild [70.17413507444704]
  We introduce QAEdit, a new benchmark aligned with widely used question answering (QA) datasets, and WILD, a task-agnostic evaluation framework.<n>Our single editing experiments show that current editing methods perform substantially worse than previously reported.
  arXiv  Detail & Related papers  (2025-02-16T15:57:55Z)
• Assessing Human Editing Effort on LLM-Generated Texts via Compression-Based Edit Distance [2.1792283995628465]
  Existing edit distance metrics, such as Levenshtein, BLEU, ROUGE, and TER, often fail to accurately measure the effort required for post-editing.<n>We introduce a novel compression-based edit distance metric grounded in the Lempel-Ziv-77 algorithm.
  arXiv  Detail & Related papers  (2024-12-23T06:29:25Z)
• Editing Arbitrary Propositions in LLMs without Subject Labels [88.67755930096966]
  We introduce a simple and fast localization method called Gradient Tracing (GT) GT allows editing arbitrary propositions instead of just binary ones, and does so without the need for subject labels. We show that our method -- without access to subject labels -- performs close to state-of-the-art L&E methods which has access subject labels.
  arXiv  Detail & Related papers  (2024-01-15T08:08:24Z)
• Beyond the Chat: Executable and Verifiable Text-Editing with LLMs [87.84199761550634]
  Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. We present InkSync, an editing interface that suggests executable edits directly within the document being edited.
  arXiv  Detail & Related papers  (2023-09-27T00:56:17Z)
• ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews [36.76200047558003]
  We introduce the task of automatically revising scientific papers based on peer feedback. We provide labels linking each reviewer comment to the specific paper edits made by the author in response. In experiments with 10 models covering the state of the art, we find that they struggle even to identify which edits correspond to a comment.
  arXiv  Detail & Related papers  (2023-06-21T22:00:03Z)
• SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages [87.08880616654258]
  We introduce the SWiPE dataset, which reconstructs the document-level editing process from English Wikipedia (EW) articles to paired Simple Wikipedia (SEW) articles. We work with Wikipedia editors to annotate 5,000 EW-SEW document pairs, labeling more than 40,000 edits with proposed 19 categories. We find that SWiPE-trained models generate more complex edits while reducing unwanted edits.
  arXiv  Detail & Related papers  (2023-05-30T16:52:42Z)
• Memory-Based Model Editing at Scale [102.28475739907498]
  Existing model editors struggle to accurately model an edit's intended scope. We propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC) SERAC stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed.
  arXiv  Detail & Related papers  (2022-06-13T23:40:34Z)
• Learning Structural Edits via Incremental Tree Transformations [102.64394890816178]
  We present a generic model for incremental editing of structured data (i.e., "structural edits") Our editor learns to iteratively generate tree edits (e.g., deleting or adding a subtree) and applies them to the partially edited data. We evaluate our proposed editor on two source code edit datasets, where results show that, with the proposed edit encoder, our editor significantly improves accuracy over previous approaches.
  arXiv  Detail & Related papers  (2021-01-28T16:11:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.