Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction

• URL: http://arxiv.org/abs/2309.11439v1
• Date: Wed, 20 Sep 2023 16:14:10 GMT
• Title: Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction
• Authors: Masahiro Kaneko, Naoaki Okazaki
• Abstract summary: It is crucial to ensure the user's comprehension of a reason for correction. Existing studies present tokens, examples, and hints as to the basis for correction but do not directly explain the reasons for corrections. Generating explanations for GEC corrections involves aligning input and output tokens, identifying correction points, and presenting corresponding explanations consistently. This study introduces a method called controlled generation with Prompt Insertion (PI) so that LLMs can explain the reasons for corrections in natural language.
• Score: 50.66922361766939
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In Grammatical Error Correction (GEC), it is crucial to ensure the user's comprehension of a reason for correction. Existing studies present tokens, examples, and hints as to the basis for correction but do not directly explain the reasons for corrections. Although methods that use Large Language Models (LLMs) to provide direct explanations in natural language have been proposed for various tasks, no such method exists for GEC. Generating explanations for GEC corrections involves aligning input and output tokens, identifying correction points, and presenting corresponding explanations consistently. However, it is not straightforward to specify a complex format to generate explanations, because explicit control of generation is difficult with prompts. This study introduces a method called controlled generation with Prompt Insertion (PI) so that LLMs can explain the reasons for corrections in natural language. In PI, LLMs first correct the input text, and then we automatically extract the correction points based on the rules. The extracted correction points are sequentially inserted into the LLM's explanation output as prompts, guiding the LLMs to generate explanations for the correction points. We also create an Explainable GEC (XGEC) dataset of correction reasons by annotating NUCLE, CoNLL2013, and CoNLL2014. Although generations from GPT-3 and ChatGPT using original prompts miss some correction points, the generation control using PI can explicitly guide to describe explanations for all correction points, contributing to improved performance in generating correction reasons.

Related papers

• EXCGEC: A Benchmark of Edit-wise Explainable Chinese Grammatical Error Correction [21.869368698234247]
  This paper introduces the task of EXplainable GEC (EXGEC), which focuses on the integral role of both correction and explanation tasks. We propose EXCGEC, a tailored benchmark for Chinese EXGEC consisting of 8,216 explanation-augmented samples.
  arXiv  Detail & Related papers  (2024-07-01T03:06:41Z)
• LM-Combiner: A Contextual Rewriting Model for Chinese Grammatical Error Correction [49.0746090186582]
  Over-correction is a critical problem in Chinese grammatical error correction (CGEC) task. Recent work using model ensemble methods can effectively mitigate over-correction and improve the precision of the GEC system. We propose the LM-Combiner, a rewriting model that can directly modify the over-correction of GEC system outputs without a model ensemble.
  arXiv  Detail & Related papers  (2024-03-26T06:12:21Z)
• Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector [25.450566841158864]
  Chinese grammatical error correction (CGEC) faces serious overcorrection challenges when employing autoregressive generative models. We propose an alignment-enhanced corrector for the overcorrection problem. Experimental results on three CGEC datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-02-07T05:56:54Z)
• Enhancing Grammatical Error Correction Systems with Explanations [45.69642286275681]
  Grammatical error correction systems improve written communication by detecting and correcting language mistakes. We introduce EXPECT, a dataset annotated with evidence words and grammatical error types. Human evaluation verifies our explainable GEC system's explanations can assist second-language learners in determining whether to accept a correction suggestion.
  arXiv  Detail & Related papers  (2023-05-25T03:00:49Z)
• GRACE: Discriminator-Guided Chain-of-Thought Reasoning [75.35436025709049]
  We propose Guiding chain-of-thought ReAsoning with a CorrectnEss Discriminator (GRACE) to steer the decoding process towards producing correct reasoning steps. GRACE employs a discriminator trained with a contrastive loss over correct and incorrect steps, which is used during decoding to score next-step candidates.
  arXiv  Detail & Related papers  (2023-05-24T09:16:51Z)
• Reducing Sequence Length by Predicting Edit Operations with Large Language Models [50.66922361766939]
  This paper proposes predicting edit spans for the source text for local sequence transduction tasks. We apply instruction tuning for Large Language Models on the supervision data of edit spans. Experiments show that the proposed method achieves comparable performance to the baseline in four tasks.
  arXiv  Detail & Related papers  (2023-05-19T17:51:05Z)
• Interpretability for Language Learners Using Example-Based Grammatical Error Correction [27.850970793739933]
  We introduce an Example-Based GEC (EB-GEC) that presents examples to language learners as a basis for a correction result. Experiments demonstrate that the examples presented by EB-GEC help language learners decide to accept or refuse suggestions from the GEC output.
  arXiv  Detail & Related papers  (2022-03-14T13:15:00Z)
• Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction [49.25830718574892]
  We present a new framework named Tail-to-Tail (textbfTtT) non-autoregressive sequence prediction. Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected. Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure.
  arXiv  Detail & Related papers  (2021-06-03T05:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.