RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation

• URL: http://arxiv.org/abs/2310.07299v1
• Date: Wed, 11 Oct 2023 08:33:23 GMT
• Title: RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation
• Authors: Yue Zhang, Leyang Cui, Enbo Zhao, Wei Bi, Shuming Shi
• Abstract summary: We introduce RobustGEC, a benchmark designed to evaluate the context robustness of GEC systems. We reveal that state-of-the-art GEC systems still lack sufficient robustness against context perturbations.
• Score: 64.2568239429946
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Grammatical Error Correction (GEC) systems play a vital role in assisting people with their daily writing tasks. However, users may sometimes come across a GEC system that initially performs well but fails to correct errors when the inputs are slightly modified. To ensure an ideal user experience, a reliable GEC system should have the ability to provide consistent and accurate suggestions when encountering irrelevant context perturbations, which we refer to as context robustness. In this paper, we introduce RobustGEC, a benchmark designed to evaluate the context robustness of GEC systems. RobustGEC comprises 5,000 GEC cases, each with one original error-correct sentence pair and five variants carefully devised by human annotators. Utilizing RobustGEC, we reveal that state-of-the-art GEC systems still lack sufficient robustness against context perturbations. In addition, we propose a simple yet effective method for remitting this issue.

Related papers

• CLEME2.0: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction [28.533044857379647]
  The paper focuses on improving the interpretability of Grammatical Error Correction (GEC) metrics. We propose CLEME2.0, a reference-based evaluation strategy that can describe four elementary dimensions of GEC systems.
  arXiv  Detail & Related papers  (2024-07-01T03:35:58Z)
• LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction [49.0746090186582]
  Over-correction is a critical problem in Chinese grammatical error correction (CGEC) task. Recent work using model ensemble methods can effectively mitigate over-correction and improve the precision of the GEC system. We propose the LM-Combiner, a rewriting model that can directly modify the over-correction of GEC system outputs without a model ensemble.
  arXiv  Detail & Related papers  (2024-03-26T06:12:21Z)
• Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correction [53.55440811942249]
  Grammatical Error Correction (GEC) is the task of correcting errorful sentences into grammatically correct, semantically consistent, and coherent sentences. We propose the AMR-GEC, a seq-to-seq model that incorporates denoised AMR as additional knowledge.
  arXiv  Detail & Related papers  (2023-07-05T09:06:56Z)
• FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction [6.116341682577877]
  Grammatical Error Correction (GEC) has been broadly applied in automatic correction and proofreading system recently. We present FCGEC, a fine-grained corpus to detect, identify and correct the grammatical errors.
  arXiv  Detail & Related papers  (2022-10-22T06:29:05Z)
• Gender Bias and Universal Substitution Adversarial Attacks on Grammatical Error Correction Systems for Automated Assessment [1.4213973379473654]
  GEC systems are often used on speech transcriptions of English learners as a form of assessment and feedback. The count of edits from a candidate's input sentence to a GEC system's grammatically corrected output sentence is indicative of a candidate's language ability. This work examines a simple universal substitution adversarial attack that non-native speakers of English could realistically employ to deceive GEC systems used for assessment.
  arXiv  Detail & Related papers  (2022-08-19T17:44:13Z)
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction [83.14159143179269]
  Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in sentences. We propose a syntax-guided GEC model (SG-GEC) which adopts the graph attention mechanism to utilize the syntactic knowledge of dependency trees. We evaluate our model on public benchmarks of GEC task and it achieves competitive results.
  arXiv  Detail & Related papers  (2021-11-05T07:07:48Z)
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction [128.9174409251852]
  We show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector.
  arXiv  Detail & Related papers  (2021-09-14T17:06:43Z)
• Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction [98.31440090585376]
  Grammatical Error Correction (GEC) aims to correct writing errors and help language learners improve their writing skills. Existing GEC models tend to produce spurious corrections or fail to detect lots of errors. This paper presents the Neural Verification Network (VERNet) for GEC quality estimation with multiple hypotheses.
  arXiv  Detail & Related papers  (2021-05-10T15:04:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.