StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions

• URL: http://arxiv.org/abs/2407.12423v3
• Date: Tue, 17 Sep 2024 08:32:02 GMT
• Title: StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions
• Authors: Zixin Chen, Jiachen Wang, Meng Xia, Kento Shigyo, Dingdong Liu, Rong Zhang, Huamin Qu,
• Abstract summary: We present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT.
• Score: 35.927734064685886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students' learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students' interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master's level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students' interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system's effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.

Related papers

• ChatGPT in Research and Education: Exploring Benefits and Threats [1.9466452723529557]
  ChatGPT is a powerful language model developed by OpenAI. It offers personalized feedback, enhances accessibility, enables interactive conversations, assists with lesson preparation and evaluation, and introduces new methods for teaching complex subjects. ChatGPT also poses challenges to traditional education and research systems. These challenges include the risk of cheating on online exams, the generation of human-like text that may compromise academic integrity, and difficulties in assessing the reliability of information generated by AI.
  arXiv  Detail & Related papers  (2024-11-05T05:29:00Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues [53.52699766206808]
  We present a first attempt at performing knowledge tracing (KT) in tutor-student dialogues. We propose methods to identify the knowledge components/skills involved in each dialogue turn. We then apply a range of KT methods on the resulting labeled data to track student knowledge levels over an entire dialogue.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• ChatGPT in Data Visualization Education: A Student Perspective [19.58123915686711]
  This work explores the impact of such technology on student learning in an interdisciplinary, project-oriented data visualization course. Students engaged with ChatGPT across four distinct projects, designing and implementing data visualizations using a variety of tools such as Tableau, D3, and Vega-lite. Our analysis examined the advantages and barriers of using ChatGPT, students' querying behavior, the types of assistance sought, and its impact on assignment outcomes and engagement.
  arXiv  Detail & Related papers  (2024-05-01T02:40:20Z)
• Investigation of the effectiveness of applying ChatGPT in Dialogic Teaching Using Electroencephalography [6.34494999013996]
  Large language models (LLMs) possess the capability to interpret knowledge, answer questions, and consider context. This research recruited 34 undergraduate students as participants, who were randomly divided into two groups. The experimental group engaged in dialogic teaching using ChatGPT, while the control group interacted with human teachers.
  arXiv  Detail & Related papers  (2024-03-25T12:23:12Z)
• Exploring the Impact of ChatGPT on Student Interactions in Computer-Supported Collaborative Learning [1.5961625979922607]
  This paper takes an initial step in exploring the applicability of ChatGPT in a computer-supported collaborative learning environment. Using statistical analysis, we validate the shifts in student interactions during an asynchronous group brainstorming session by introducing ChatGPT as an instantaneous question-answering agent.
  arXiv  Detail & Related papers  (2024-03-11T18:18:18Z)
• Integrating ChatGPT in a Computer Science Course: Students Perceptions and Suggestions [0.0]
  This experience report explores students' perceptions and suggestions for integrating ChatGPT in a computer science course. Findings show the importance of carefully balancing using ChatGPT in computer science courses.
  arXiv  Detail & Related papers  (2023-12-22T10:48:34Z)
• Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study [51.079100495163736]
  This paper systematically inspects ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing. ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.
  arXiv  Detail & Related papers  (2023-05-15T07:14:41Z)
• A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding [55.37338324658501]
  Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data. In this work, we investigate the understanding ability of ChatGPT for zero-shot dialogue understanding tasks.
  arXiv  Detail & Related papers  (2023-04-09T15:28:36Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.