Chinese Grammatical Correction Using BERT-based Pre-trained Model

• URL: http://arxiv.org/abs/2011.02093v1
• Date: Wed, 4 Nov 2020 01:23:30 GMT
• Title: Chinese Grammatical Correction Using BERT-based Pre-trained Model
• Authors: Hongfei Wang, Michiki Kurosawa, Satoru Katsumata, and Mamoru Komachi
• Abstract summary: We verify the effectiveness of two methods that incorporate a BERT-based pre-trained model into an encoder-decoder model on Chinese grammatical error correction tasks. We also analyze the error type and conclude that sentence-level errors are yet to be addressed.
• Score: 17.847005759631703
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, pre-trained models have been extensively studied, and several downstream tasks have benefited from their utilization. In this study, we verify the effectiveness of two methods that incorporate a BERT-based pre-trained model developed by Cui et al. (2020) into an encoder-decoder model on Chinese grammatical error correction tasks. We also analyze the error type and conclude that sentence-level errors are yet to be addressed.

Related papers

• Subtle Errors Matter: Preference Learning via Error-injected Self-editing [59.405145971637204]
  We propose a novel preference learning framework called eRror-Injected Self-Editing (RISE) RISE injects predefined subtle errors into partial tokens of correct solutions to construct hard pairs for error mitigation. Experiments validate the effectiveness of RISE, with preference learning on Qwen2-7B-Instruct yielding notable improvements of 3.0% on GSM8K and 7.9% on MATH.
  arXiv  Detail & Related papers  (2024-10-09T07:43:38Z)
• EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correction [0.0]
  Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in Chinese sentences caused by phonetic or visual similarities. We propose two data augmentation methods to address these limitations. Firstly, we augment the dataset by either splitting long sentences into shorter ones or reducing typos in sentences with multiple typos.
  arXiv  Detail & Related papers  (2024-09-08T14:29:10Z)
• Chinese Spelling Correction as Rephrasing Language Model [63.65217759957206]
  We study Chinese Spelling Correction (CSC), which aims to detect and correct the potential spelling errors in a given sentence. Current state-of-the-art methods regard CSC as a sequence tagging task and fine-tune BERT-based models on sentence pairs. We propose Rephrasing Language Model (ReLM), where the model is trained to rephrase the entire sentence by infilling additional slots, instead of character-to-character tagging.
  arXiv  Detail & Related papers  (2023-08-17T06:04:28Z)
• Does Correction Remain A Problem For Large Language Models? [63.24433996856764]
  This paper investigates the role of correction in the context of large language models by conducting two experiments. The first experiment focuses on correction as a standalone task, employing few-shot learning techniques with GPT-like models for error correction. The second experiment explores the notion of correction as a preparatory task for other NLP tasks, examining whether large language models can tolerate and perform adequately on texts containing certain levels of noise or errors.
  arXiv  Detail & Related papers  (2023-08-03T14:09:31Z)
• An Error-Guided Correction Model for Chinese Spelling Error Correction [13.56600372085612]
  We propose an error-guided correction model (EGCM) to improve Chinese spelling correction. Our model achieves superior performance against state-of-the-art approaches by a remarkable margin.
  arXiv  Detail & Related papers  (2023-01-16T09:27:45Z)
• Improving Pre-trained Language Models with Syntactic Dependency Prediction Task for Chinese Semantic Error Recognition [52.55136323341319]
  Existing Chinese text error detection mainly focuses on spelling and simple grammatical errors. Chinese semantic errors are understudied and more complex that humans cannot easily recognize.
  arXiv  Detail & Related papers  (2022-04-15T13:55:32Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)
• Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors [3.55517579369797]
  We show that 5 to 10% of training data are enough for a BERT-based error detection method to achieve performance equivalent to a non-language model-based method. We also show with pseudo error data that it actually exhibits such nice properties in learning rules for recognizing various types of error.
  arXiv  Detail & Related papers  (2021-08-27T10:37:14Z)
• Translation Error Detection as Rationale Extraction [36.616561917049076]
  We study the behaviour of state-of-the-art sentence-level QE models and show that explanations can indeed be used to detect translation errors. We introduce a novel semi-supervised method for word-level QE and (ii) propose to use the QE task as a new benchmark for evaluating the plausibility of feature attribution.
  arXiv  Detail & Related papers  (2021-08-27T09:35:14Z)
• Grammatical Error Generation Based on Translated Fragments [0.0]
  We perform neural machine translation of sentence fragments in order to create large amounts of training data for English grammatical error correction. Our method aims at simulating mistakes made by second language learners, and produces a wider range of non-native style language.
  arXiv  Detail & Related papers  (2021-04-20T12:43:40Z)
• On the Robustness of Language Encoders against Grammatical Errors [66.05648604987479]
  We collect real grammatical errors from non-native speakers and conduct adversarial attacks to simulate these errors on clean text data. Results confirm that the performance of all tested models is affected but the degree of impact varies.
  arXiv  Detail & Related papers  (2020-05-12T11:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.