Scoring Edit Impact in Grammatical Error Correction via Embedded Association Graphs

• URL: http://arxiv.org/abs/2604.06573v1
• Date: Wed, 08 Apr 2026 01:37:59 GMT
• Title: Scoring Edit Impact in Grammatical Error Correction via Embedded Association Graphs
• Abstract summary: Grammatical Error Correction (GEC) systems produce a sequence of edits to correct an erroneous sentence.<n>The quality of these edits is typically evaluated against human annotations.<n>Recent meta-evaluation approaches rely on human judgments across multiple references, but they are difficult to scale to large datasets.<n>We propose a new task, Scoring Edit Impact in GEC, which aims to automatically estimate the importance of edits produced by a GEC system.
• Score: 12.608012136037997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A Grammatical Error Correction (GEC) system produces a sequence of edits to correct an erroneous sentence. The quality of these edits is typically evaluated against human annotations. However, a sentence may admit multiple valid corrections, and existing evaluation settings do not fully accommodate diverse application scenarios. Recent meta-evaluation approaches rely on human judgments across multiple references, but they are difficult to scale to large datasets. In this paper, we propose a new task, Scoring Edit Impact in GEC, which aims to automatically estimate the importance of edits produced by a GEC system. To address this task, we introduce a scoring framework based on an embedded association graph. The graph captures latent dependencies among edits and syntactically related edits, grouping them into coherent groups. We then perform perplexity-based scoring to estimate each edit's contribution to sentence fluency. Experiments across 4 GEC datasets, 4 languages, and 4 GEC systems demonstrate that our method consistently outperforms a range of baselines. Further analysis shows that the embedded association graph effectively captures cross-linguistic structural dependencies among edits.

Related papers

• CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction [15.181557665760161]
  Grammatical error correction (GEC) systems are usually trained and evaluated on GEC benchmarks.<n>Existing GEC models usually fail to understand the error patterns in the varying contexts.<n>We propose CoCoGEC, a counterfactual generation framework that creates copies of training instances with error-irrelevant contexts altered.
  arXiv  Detail & Related papers  (2026-06-13T02:58:13Z)
• Grammatical Error Correction Evaluation by Optimally Transporting Edit Representation [34.071151696990384]
  This study focuses on edits specifically designed for grammatical error correction (GEC)<n>We propose edit vector, a representation for an edit, and introduce a new metric, UOT-ERRANT, which transports these edit vectors from hypothesis to reference using unbalanced optimal transport.<n>Experiments with SEEDA meta-evaluation show that UOT-ERRANT improves evaluation performance, particularly in the +Fluency domain.
  arXiv  Detail & Related papers  (2026-02-05T08:05:42Z)
• ChartE$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing [64.65742943745866]
  ChartE$3$ is an End-to-End Chart Editing benchmark.<n>It directly evaluates models without relying on intermediate natural language programs or code-level supervision.<n>It contains over 1,200 high-quality samples constructed via a well-designed data pipeline with human curation.
  arXiv  Detail & Related papers  (2026-01-29T13:29:27Z)
• Charts Are Not Images: On the Challenges of Scientific Chart Editing [66.38730113476677]
  textitFigEdit is a benchmark for scientific figure editing comprising over 30,000 samples.<n>Our benchmark demonstrates the profound limitations of pixel-level manipulation.<n>By releasing textitFigEdit, we aim to enable systematic progress in structure-aware figure editing.
  arXiv  Detail & Related papers  (2025-11-30T06:13:48Z)
• Improving Explainability of Sentence-level Metrics via Edit-level Attribution for Grammatical Error Correction [11.512856112792093]
  We propose attributing sentence-level scores to individual edits, providing insight into how specific corrections contribute to the overall performance.<n> Experiments with existing sentence-level metrics demonstrate high consistency across different edit granularities and show approximately 70% alignment with human evaluations.<n>In addition, we analyze biases in the metrics based on the attribution results, revealing trends such as the tendency to ignore orthographic edits.
  arXiv  Detail & Related papers  (2024-12-17T17:31:17Z)
• Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation [73.9145653659403]
  We show that Generative Error Correction models struggle to generalize beyond the specific types of errors encountered during training. We propose DARAG, a novel approach designed to improve GEC for ASR in in-domain (ID) and OOD scenarios. Our approach is simple, scalable, and both domain- and language-agnostic.
  arXiv  Detail & Related papers  (2024-10-17T04:00:29Z)
• Revisiting Meta-evaluation for Grammatical Error Correction [14.822205658480813]
  SEEDA is a new dataset for GEC meta-evaluation. It consists of corrections with human ratings along two different granularities. The results suggest that edit-based metrics may have been underestimated in existing studies.
  arXiv  Detail & Related papers  (2024-03-05T05:53:09Z)
• CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction [32.44051877804761]
  Chunk-LEvel Multi-reference Evaluation (CLEME) is designed to evaluate Grammatical Error Correction (GEC) systems in the multi-reference evaluation setting. We conduct experiments on six English reference sets based on the CoNLL-2014 shared task.
  arXiv  Detail & Related papers  (2023-05-18T08:57:17Z)
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction [83.14159143179269]
  Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in sentences. We propose a syntax-guided GEC model (SG-GEC) which adopts the graph attention mechanism to utilize the syntactic knowledge of dependency trees. We evaluate our model on public benchmarks of GEC task and it achieves competitive results.
  arXiv  Detail & Related papers  (2021-11-05T07:07:48Z)
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction [128.9174409251852]
  We show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector.
  arXiv  Detail & Related papers  (2021-09-14T17:06:43Z)
• A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction [54.569707226277735]
  Existing approaches for grammatical error correction (GEC) rely on supervised learning with manually created GEC datasets. There is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. We propose a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models.
  arXiv  Detail & Related papers  (2020-10-07T04:45:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.