Exploring the Efficacy of ChatGPT in Analyzing Student Teamwork Feedback with an Existing Taxonomy

• URL: http://arxiv.org/abs/2305.11882v1
• Date: Tue, 9 May 2023 19:55:50 GMT
• Title: Exploring the Efficacy of ChatGPT in Analyzing Student Teamwork Feedback with an Existing Taxonomy
• Authors: Andrew Katz, Siqing Wei, Gaurav Nanda, Christopher Brinton, Matthew Ohland
• Abstract summary: ChatGPT can achieve over 90% accuracy in labeling student comments. This study contributes to the growing body of research on the use of AI models in educational contexts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Teamwork is a critical component of many academic and professional settings. In those contexts, feedback between team members is an important element to facilitate successful and sustainable teamwork. However, in the classroom, as the number of teams and team members and frequency of evaluation increase, the volume of comments can become overwhelming for an instructor to read and track, making it difficult to identify patterns and areas for student improvement. To address this challenge, we explored the use of generative AI models, specifically ChatGPT, to analyze student comments in team based learning contexts. Our study aimed to evaluate ChatGPT's ability to accurately identify topics in student comments based on an existing framework consisting of positive and negative comments. Our results suggest that ChatGPT can achieve over 90\% accuracy in labeling student comments, providing a potentially valuable tool for analyzing feedback in team projects. This study contributes to the growing body of research on the use of AI models in educational contexts and highlights the potential of ChatGPT for facilitating analysis of student comments.

Related papers

• ChatGPT in Research and Education: Exploring Benefits and Threats [1.9466452723529557]
  ChatGPT is a powerful language model developed by OpenAI. It offers personalized feedback, enhances accessibility, enables interactive conversations, assists with lesson preparation and evaluation, and introduces new methods for teaching complex subjects. ChatGPT also poses challenges to traditional education and research systems. These challenges include the risk of cheating on online exams, the generation of human-like text that may compromise academic integrity, and difficulties in assessing the reliability of information generated by AI.
  arXiv  Detail & Related papers  (2024-11-05T05:29:00Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [50.067342343957876]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL) Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Integrating ChatGPT in a Computer Science Course: Students Perceptions and Suggestions [0.0]
  This experience report explores students' perceptions and suggestions for integrating ChatGPT in a computer science course. Findings show the importance of carefully balancing using ChatGPT in computer science courses.
  arXiv  Detail & Related papers  (2023-12-22T10:48:34Z)
• "Close...but not as good as an educator." -- Using ChatGPT to provide formative feedback in large-class collaborative learning [0.0]
  We employed ChatGPT to provide personalised formative feedback in a one-hour Zoom break-out room activity. Half of the 44 survey respondents had never used ChatGPT before. Only three groups used the feedback loop to improve their evaluation plans.
  arXiv  Detail & Related papers  (2023-11-02T23:00:38Z)
• Evaluation of ChatGPT Feedback on ELL Writers' Coherence and Cohesion [0.7028778922533686]
  ChatGPT has had a transformative effect on education where students are using it to help with homework assignments and teachers are actively employing it in their teaching practices. This study evaluated the quality of the feedback generated by ChatGPT regarding the coherence and cohesion of the essays written by English Language learners (ELLs) students.
  arXiv  Detail & Related papers  (2023-10-10T10:25:56Z)
• Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study [51.079100495163736]
  This paper systematically inspects ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing. ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.
  arXiv  Detail & Related papers  (2023-05-15T07:14:41Z)
• ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions [5.962828109329824]
  ChatGPT is an AI language model developed by OpenAI that can understand and generate human-like text. There is concern that students may leverage ChatGPT to complete take-home assignments and exams and obtain favorable grades without genuinely acquiring knowledge.
  arXiv  Detail & Related papers  (2023-04-28T17:26:32Z)
• A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding [55.37338324658501]
  Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data. In this work, we investigate the understanding ability of ChatGPT for zero-shot dialogue understanding tasks.
  arXiv  Detail & Related papers  (2023-04-09T15:28:36Z)
• Is ChatGPT a Good NLG Evaluator? A Preliminary Study [121.77986688862302]
  We provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.
  arXiv  Detail & Related papers  (2023-03-07T16:57:20Z)
• On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective [67.98821225810204]
  We evaluate the robustness of ChatGPT from the adversarial and out-of-distribution perspective. Results show consistent advantages on most adversarial and OOD classification and translation tasks. ChatGPT shows astounding performance in understanding dialogue-related texts.
  arXiv  Detail & Related papers  (2023-02-22T11:01:20Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.