RECIPE4U: Student-ChatGPT Interaction Dataset in EFL Writing Education
- URL: http://arxiv.org/abs/2403.08272v1
- Date: Wed, 13 Mar 2024 05:51:57 GMT
- Title: RECIPE4U: Student-ChatGPT Interaction Dataset in EFL Writing Education
- Authors: Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon
Ahn and Alice Oh
- Abstract summary: We present RECIPE4U, a dataset sourced from a semester-long experiment with 212 college students in English as Foreign Language (EFL) writing courses.
During the study, students engaged in dialogues with ChatGPT to revise their essays. RECIPE4U includes comprehensive records of these interactions, including conversation logs, students' intent, students' self-rated satisfaction, and students' essay edit histories.
- Score: 15.253081304714101
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: The integration of generative AI in education is expanding, yet empirical
analyses of large-scale and real-world interactions between students and AI
systems still remain limited. Addressing this gap, we present RECIPE4U (RECIPE
for University), a dataset sourced from a semester-long experiment with 212
college students in English as Foreign Language (EFL) writing courses. During
the study, students engaged in dialogues with ChatGPT to revise their essays.
RECIPE4U includes comprehensive records of these interactions, including
conversation logs, students' intent, students' self-rated satisfaction, and
students' essay edit histories. In particular, we annotate the students'
utterances in RECIPE4U with 13 intention labels based on our coding schemes. We
establish baseline results for two subtasks in task-oriented dialogue systems
within educational contexts: intent detection and satisfaction estimation. As a
foundational step, we explore student-ChatGPT interaction patterns through
RECIPE4U and analyze them by focusing on students' dialogue, essay data
statistics, and students' essay edits. We further illustrate potential
applications of RECIPE4U dataset for enhancing the incorporation of LLMs in
educational frameworks. RECIPE4U is publicly available at
https://zeunie.github.io/RECIPE4U/.
Related papers
- Exploring Knowledge Tracing in Tutor-Student Dialogues [53.52699766206808]
We present a first attempt at performing knowledge tracing (KT) in tutor-student dialogues.
We propose methods to identify the knowledge components/skills involved in each dialogue turn.
We then apply a range of KT methods on the resulting labeled data to track student knowledge levels over an entire dialogue.
arXiv Detail & Related papers (2024-09-24T22:31:39Z) - GPT-4 as a Homework Tutor can Improve Student Engagement and Learning Outcomes [80.60912258178045]
We developed a prompting strategy that enables GPT-4 to conduct interactive homework sessions for high-school students learning English as a second language.
We carried out a Randomized Controlled Trial (RCT) in four high-school classes, replacing traditional homework with GPT-4 homework sessions for the treatment group.
We observed significant improvements in learning outcomes, specifically a greater gain in grammar, and student engagement.
arXiv Detail & Related papers (2024-09-24T11:22:55Z) - Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learning [3.2721068185888127]
We present a proof-of-concept application to offer students dynamic and contextualized feedback.
Specifically, we augment an Online Programming Exercise bot for a college-level Cloud Computing course with ChatGPT.
We demonstrate that LLMs can be used to generate highly situated reflection triggers that incorporate details of the collaborative discussion happening in context.
arXiv Detail & Related papers (2024-04-28T17:56:14Z) - Learning from Emotions, Demographic Information and Implicit User
Feedback in Task-Oriented Document-Grounded Dialogues [59.516187851808375]
We introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback.
Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.
arXiv Detail & Related papers (2024-01-17T14:52:26Z) - Learning From Free-Text Human Feedback -- Collect New Datasets Or Extend
Existing Ones? [57.16050211534735]
We investigate the types and frequency of free-text human feedback in commonly used dialog datasets.
Our findings provide new insights into the composition of the datasets examined, including error types, user response types, and the relations between them.
arXiv Detail & Related papers (2023-10-24T12:01:11Z) - ChEDDAR: Student-ChatGPT Dialogue in EFL Writing Education [14.524728335166703]
The integration of generative AI in education is expanding, yet empirical analyses of large-scale, real-world interactions between students and AI systems remain limited.
This study collects a semester-long longitudinal experiment involving 212 college students enrolled in English as Foreign Langauge (EFL) writing courses.
ChEDDAR includes a conversation log, utterance-level essay edit history, self-rated satisfaction, and students' intent, in addition to session-level pre-and-post surveys documenting their objectives and overall experiences.
arXiv Detail & Related papers (2023-09-23T03:28:25Z) - Analysis of Individual Conversational Volatility in Tandem
Telecollaboration for Second Language Learning [7.767962338247332]
Students are grouped into video conference calls while learning the native language of other student(s) on the calls.
This places students in an online environment where the more outgoing can actively contribute and engage in dialogue.
We have built and deployed the L2L system which records timings of conversational utterances from all participants in a call.
We present an analysis of conversational volatility measures for a sample of 19 individual English-speaking students from our University who are learning Frenchm, in each of 86 tandem telecollaboration calls over one teaching semester.
arXiv Detail & Related papers (2022-06-28T12:34:00Z) - UniDU: Towards A Unified Generative Dialogue Understanding Framework [62.8474841241855]
We investigate a unified generative dialogue understanding framework, namely UniDU, to achieve information exchange among DU tasks.
We conduct experiments on ten dialogue understanding datasets, which span five fundamental tasks.
The proposed UniDU framework outperforms task-specific well-designed methods on all 5 tasks.
arXiv Detail & Related papers (2022-04-10T09:32:34Z) - Structural Pre-training for Dialogue Comprehension [51.215629336320305]
We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features.
To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives.
Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
arXiv Detail & Related papers (2021-05-23T15:16:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.