Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation

• URL: http://arxiv.org/abs/2309.01645v1
• Date: Mon, 4 Sep 2023 14:54:39 GMT
• Title: Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation
• Authors: Siyi Cao, Linping Zhong
• Abstract summary: ChatGPT, a cutting-edge AI-powered,can quickly generate responses on given commands. This study compared the revised Chinese to English translation texts produced by Chinese Master of Translation and Interpretation (MTI) students.
• Score: 1.25097469793837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: ChatGPT,a cutting-edge AI-powered Chatbot,can quickly generate responses on given commands. While it was reported that ChatGPT had the capacity to deliver useful feedback, it is still unclear about its effectiveness compared with conventional feedback approaches,such as teacher feedback (TF) and self-feedback (SF). To address this issue, this study compared the revised Chinese to English translation texts produced by Chinese Master of Translation and Interpretation (MTI) students,who learned English as a Second/Foreign Language (ESL/EFL), based on three feedback types (i.e., ChatGPT-based feedback, TF and SF). The data was analyzed using BLEU score to gauge the overall translation quality as well as Coh-Metrix to examine linguistic features across three dimensions: lexicon, syntax, and cohesion.The findings revealed that TF- and SF-guided translation texts surpassed those with ChatGPT-based feedback, as indicated by the BLEU score. In terms of linguistic features,ChatGPT-based feedback demonstrated superiority, particularly in enhancing lexical capability and referential cohesion in the translation texts. However, TF and SF proved more effective in developing syntax-related skills,as it addressed instances of incorrect usage of the passive voice. These diverse outcomes indicate ChatGPT's potential as a supplementary resource, complementing traditional teacher-led methods in translation practice.

Related papers

• What is the Best Way for ChatGPT to Translate Poetry? [38.47691441569612]
  This study examines ChatGPT's capabilities in English-Chinese poetry translation tasks, utilizing targeted prompts and small sample scenarios to ascertain optimal performance. We propose an Explanation-Assisted Poetry Machine Translation (EAPMT) method, which leverages monolingual poetry explanation as a guiding information for the translation process. The results from both human and machine evaluations demonstrate that our EAPMT method outperforms traditional translation methods of ChatGPT and the existing online systems.
  arXiv  Detail & Related papers  (2024-06-05T16:48:26Z)
• Evaluation of ChatGPT Feedback on ELL Writers' Coherence and Cohesion [0.7028778922533686]
  ChatGPT has had a transformative effect on education where students are using it to help with homework assignments and teachers are actively employing it in their teaching practices. This study evaluated the quality of the feedback generated by ChatGPT regarding the coherence and cohesion of the essays written by English Language learners (ELLs) students.
  arXiv  Detail & Related papers  (2023-10-10T10:25:56Z)
• Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text [48.36706154871577]
  We introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts) It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. We also propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text.
  arXiv  Detail & Related papers  (2023-07-21T06:38:37Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback [90.20262941911027]
  ParroT is a framework to enhance and regulate the translation abilities during chat. Specifically, ParroT reformulates translation data into the instruction-following style. We propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction.
  arXiv  Detail & Related papers  (2023-04-05T13:12:00Z)
• How to Design Translation Prompts for ChatGPT: An Empirical Study [18.678893287863033]
  ChatGPT has demonstrated surprising abilities in natural language understanding and natural language generation. We adopt several translation prompts on a wide range of translations. Our work provides empirical evidence that ChatGPT still has great potential in translations.
  arXiv  Detail & Related papers  (2023-04-05T01:17:59Z)
• ChatGPT as a Factual Inconsistency Evaluator for Text Summarization [17.166794984161964]
  We show that ChatGPT can evaluate factual inconsistency under a zero-shot setting. It generally outperforms previous evaluation metrics on binary entailment inference, summary ranking, and consistency rating. However, a closer inspection of ChatGPT's output reveals certain limitations including its preference for more lexically similar candidates, false reasoning, and inadequate understanding of instructions.
  arXiv  Detail & Related papers  (2023-03-27T22:30:39Z)
• Towards Making the Most of ChatGPT for Machine Translation [75.576405098545]
  ChatGPT shows remarkable capabilities for machine translation (MT) Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages.
  arXiv  Detail & Related papers  (2023-03-24T03:35:21Z)
• Chinese Intermediate English Learners outdid ChatGPT in deep cohesion: Evidence from English narrative writing [5.747170211018015]
  This study compared the writing performance on a narrative topic by ChatGPT and Chinese intermediate English (CIE) learners. Data were analyzed in terms of five discourse components using Coh-Metrix.
  arXiv  Detail & Related papers  (2023-03-21T12:55:54Z)
• Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine [97.8609714773255]
  We evaluate ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. ChatGPT performs competitively with commercial translation products but lags behind significantly on low-resource or distant languages. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted.
  arXiv  Detail & Related papers  (2023-01-20T08:51:36Z)
• Multilingual Alignment of Contextual Word Representations [49.42244463346612]
  BERT exhibits significantly improved zero-shot performance on XNLI compared to the base model. We introduce a contextual version of word retrieval and show that it correlates well with downstream zero-shot transfer. These results support contextual alignment as a useful concept for understanding large multilingual pre-trained models.
  arXiv  Detail & Related papers  (2020-02-10T03:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.