IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator

• URL: http://arxiv.org/abs/2506.02899v1
• Date: Tue, 03 Jun 2025 14:05:37 GMT
• Title: IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator
• Authors: Yusuke Sakai, Takumi Goto, Taro Watanabe,
• Abstract summary: IMPARA-GED is a novel reference-free automatic grammatical error correction (GEC) evaluation method with grammatical error detection (GED) capabilities.<n>We focus on the quality estimator of IMPARA, an existing automatic GEC evaluation method, and construct that of IMPARA-GED using a pre-trained language model with enhanced GED capabilities.
• Score: 13.02513034520894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose IMPARA-GED, a novel reference-free automatic grammatical error correction (GEC) evaluation method with grammatical error detection (GED) capabilities. We focus on the quality estimator of IMPARA, an existing automatic GEC evaluation method, and construct that of IMPARA-GED using a pre-trained language model with enhanced GED capabilities. Experimental results on SEEDA, a meta-evaluation dataset for automatic GEC evaluation methods, demonstrate that IMPARA-GED achieves the highest correlation with human sentence-level evaluations.

Related papers

• DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models [39.493913608472404]
  Large language model (LLM)-based Grammatical Error Correction (GEC) models often produce corrections that diverge from provided gold references.<n>This discrepancy undermines the reliability of traditional reference-based evaluation metrics.<n>We propose a novel evaluation framework for GEC models, DSGram, integrating Semantic Coherence, Edit Level, and Fluency, and utilizing a dynamic weighting mechanism.
  arXiv  Detail & Related papers  (2024-12-17T11:54:16Z)
• Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation [73.9145653659403]
  We show that Generative Error Correction models struggle to generalize beyond the specific types of errors encountered during training. We propose DARAG, a novel approach designed to improve GEC for ASR in in-domain (ID) and OOD scenarios. Our approach is simple, scalable, and both domain- and language-agnostic.
  arXiv  Detail & Related papers  (2024-10-17T04:00:29Z)
• LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction [49.0746090186582]
  Over-correction is a critical problem in Chinese grammatical error correction (CGEC) task. Recent work using model ensemble methods can effectively mitigate over-correction and improve the precision of the GEC system. We propose the LM-Combiner, a rewriting model that can directly modify the over-correction of GEC system outputs without a model ensemble.
  arXiv  Detail & Related papers  (2024-03-26T06:12:21Z)
• RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation [64.2568239429946]
  We introduce RobustGEC, a benchmark designed to evaluate the context robustness of GEC systems. We reveal that state-of-the-art GEC systems still lack sufficient robustness against context perturbations.
  arXiv  Detail & Related papers  (2023-10-11T08:33:23Z)
• Evaluation of really good grammatical error correction [0.0]
  Grammatical Error Correction (GEC) encompasses various models with distinct objectives. Traditional evaluation methods fail to fully capture the full range of system capabilities and objectives.
  arXiv  Detail & Related papers  (2023-08-17T13:45:35Z)
• Proficiency Matters Quality Estimation in Grammatical Error Correction [30.31557952622774]
  This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners' proficiency with the data.
  arXiv  Detail & Related papers  (2022-01-17T03:47:19Z)
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction [83.14159143179269]
  Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in sentences. We propose a syntax-guided GEC model (SG-GEC) which adopts the graph attention mechanism to utilize the syntactic knowledge of dependency trees. We evaluate our model on public benchmarks of GEC task and it achieves competitive results.
  arXiv  Detail & Related papers  (2021-11-05T07:07:48Z)
• Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction [98.31440090585376]
  Grammatical Error Correction (GEC) aims to correct writing errors and help language learners improve their writing skills. Existing GEC models tend to produce spurious corrections or fail to detect lots of errors. This paper presents the Neural Verification Network (VERNet) for GEC quality estimation with multiple hypotheses.
  arXiv  Detail & Related papers  (2021-05-10T15:04:25Z)
• A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction [54.569707226277735]
  Existing approaches for grammatical error correction (GEC) rely on supervised learning with manually created GEC datasets. There is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. We propose a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models.
  arXiv  Detail & Related papers  (2020-10-07T04:45:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.