Focus Is What You Need For Chinese Grammatical Error Correction

• URL: http://arxiv.org/abs/2210.12692v3
• Date: Thu, 27 Oct 2022 05:34:53 GMT
• Title: Focus Is What You Need For Chinese Grammatical Error Correction
• Authors: Jingheng Ye, Yinghui Li, Shirong Ma, Rui Xie, Wei Wu, Hai-Tao Zheng
• Abstract summary: We argue that even though this is a very reasonable hypothesis, it is too harsh for the intelligence of the mainstream models in this era. We propose a simple yet effective training strategy called OneTarget to improve the focus ability of the CGEC models.
• Score: 17.71297141482757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chinese Grammatical Error Correction (CGEC) aims to automatically detect and correct grammatical errors contained in Chinese text. In the long term, researchers regard CGEC as a task with a certain degree of uncertainty, that is, an ungrammatical sentence may often have multiple references. However, we argue that even though this is a very reasonable hypothesis, it is too harsh for the intelligence of the mainstream models in this era. In this paper, we first discover that multiple references do not actually bring positive gains to model training. On the contrary, it is beneficial to the CGEC model if the model can pay attention to small but essential data during the training process. Furthermore, we propose a simple yet effective training strategy called OneTarget to improve the focus ability of the CGEC models and thus improve the CGEC performance. Extensive experiments and detailed analyses demonstrate the correctness of our discovery and the effectiveness of our proposed method.

Related papers

• Loss-Aware Curriculum Learning for Chinese Grammatical Error Correction [21.82403446634522]
  Chinese grammatical error correction (CGEC) aims to detect and correct errors in the input Chinese sentences. Current approaches ignore that correction difficulty varies across different instances and treat these samples equally. We propose a multi-granularity Curriculum Learning framework to address this problem.
  arXiv  Detail & Related papers  (2024-12-31T08:11:49Z)
• Self-Improvement in Language Models: The Sharpening Mechanism [70.9248553790022]
  We offer a new perspective on the capabilities of self-improvement through a lens we refer to as sharpening. Motivated by the observation that language models are often better at verifying response quality than they are at generating correct responses, we formalize self-improvement as using the model itself as a verifier during post-training. We analyze two natural families of self-improvement algorithms based on SFT and RLHF.
  arXiv  Detail & Related papers  (2024-12-02T20:24:17Z)
• Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction [62.409807640887834]
  Chinese Grammatical Error Correction (CGEC) aims to correct all potential grammatical errors in the input sentences. LLMs' performance as correctors on CGEC remains unsatisfactory due to its challenging task focus. We rethink the roles of LLMs in the CGEC task so that they can be better utilized and explored in CGEC.
  arXiv  Detail & Related papers  (2024-02-18T01:40:34Z)
• Chinese Spelling Correction as Rephrasing Language Model [63.65217759957206]
  We study Chinese Spelling Correction (CSC), which aims to detect and correct the potential spelling errors in a given sentence. Current state-of-the-art methods regard CSC as a sequence tagging task and fine-tune BERT-based models on sentence pairs. We propose Rephrasing Language Model (ReLM), where the model is trained to rephrase the entire sentence by infilling additional slots, instead of character-to-character tagging.
  arXiv  Detail & Related papers  (2023-08-17T06:04:28Z)
• Rethinking Masked Language Modeling for Chinese Spelling Correction [70.85829000570203]
  We study Chinese Spelling Correction (CSC) as a joint decision made by two separate models: a language model and an error model. We find that fine-tuning BERT tends to over-fit the error model while under-fit the language model, resulting in poor generalization to out-of-distribution error patterns. We demonstrate that a very simple strategy, randomly masking 20% non-error tokens from the input sequence during fine-tuning is sufficient for learning a much better language model without sacrificing the error model.
  arXiv  Detail & Related papers  (2023-05-28T13:19:12Z)
• Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction [36.74272211767197]
  We propose a linguistic rules-based approach to construct large-scale CGEC training corpora with automatically generated grammatical errors. We present a challenging CGEC benchmark derived entirely from errors made by native Chinese speakers in real-world scenarios.
  arXiv  Detail & Related papers  (2022-10-19T10:20:39Z)
• Type-Driven Multi-Turn Corrections for Grammatical Error Correction [46.34114495164071]
  Grammatical Error Correction (GEC) aims to automatically detect and correct grammatical errors. Previous studies mainly focus on the data augmentation approach to combat the exposure bias. We propose a Type-Driven Multi-Turn Corrections approach for GEC.
  arXiv  Detail & Related papers  (2022-03-17T07:30:05Z)
• The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking [32.8563506271794]
  Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors. Pre-trained language models (PLMs) promote the progress of CSC task. We propose an Error-driven COntrastive Probability Optimization framework for CSC task.
  arXiv  Detail & Related papers  (2022-03-02T09:58:56Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction [83.14159143179269]
  Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in sentences. We propose a syntax-guided GEC model (SG-GEC) which adopts the graph attention mechanism to utilize the syntactic knowledge of dependency trees. We evaluate our model on public benchmarks of GEC task and it achieves competitive results.
  arXiv  Detail & Related papers  (2021-11-05T07:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.