From Words to Wisdom: Discourse Annotation and Baseline Models for Student Dialogue Understanding

• URL: http://arxiv.org/abs/2511.20547v1
• Date: Tue, 25 Nov 2025 17:46:00 GMT
• Title: From Words to Wisdom: Discourse Annotation and Baseline Models for Student Dialogue Understanding
• Authors: Farjana Sultana Mim, Shuchin Aeron, Eric Miller, Kristen Wendell,
• Abstract summary: This work introduces an annotated educational dialogue dataset of student conversations featuring knowledge construction and task production discourse.<n>We also establish baseline models for automatically predicting these discourse properties for each turn of talk within conversations, using pre-trained large language models GPT-3.5 and Llama-3.1.<n> Experimental results indicate that these state-of-the-art models perform suboptimally on this task, indicating the potential for future research.
• Score: 5.459797813771498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifying discourse features in student conversations is quite important for educational researchers to recognize the curricular and pedagogical variables that cause students to engage in constructing knowledge rather than merely completing tasks. The manual analysis of student conversations to identify these discourse features is time-consuming and labor-intensive, which limits the scale and scope of studies. Leveraging natural language processing (NLP) techniques can facilitate the automatic detection of these discourse features, offering educational researchers scalable and data-driven insights. However, existing studies in NLP that focus on discourse in dialogue rarely address educational data. In this work, we address this gap by introducing an annotated educational dialogue dataset of student conversations featuring knowledge construction and task production discourse. We also establish baseline models for automatically predicting these discourse properties for each turn of talk within conversations, using pre-trained large language models GPT-3.5 and Llama-3.1. Experimental results indicate that these state-of-the-art models perform suboptimally on this task, indicating the potential for future research.

Related papers

• IntrEx: A Dataset for Modeling Engagement in Educational Conversations [7.526860155587907]
  IntrEx is the first large dataset annotated for interestingness and expected interestingness in teacher-student interactions.<n>We employ a rigorous annotation process with over 100 second-language learners.<n>We investigate whether large language models (LLMs) can predict human interestingness judgments.
  arXiv  Detail & Related papers  (2025-09-08T13:07:35Z)
• LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understanding [0.0]
  This study develops a novel LLM-assisted automated coding approach for dialogue data.<n>We predict the code for an utterance based on dialogue-specific characteristics.<n>We also found the accuracy of act predictions was consistently higher than that of event predictions.
  arXiv  Detail & Related papers  (2025-04-28T12:31:38Z)
• WavChat: A Survey of Spoken Dialogue Models [66.82775211793547]
  Recent advancements in spoken dialogue models, exemplified by systems like GPT-4o, have captured significant attention in the speech domain. These advanced spoken dialogue models not only comprehend audio, music, and other speech-related features, but also capture stylistic and timbral characteristics in speech. Despite the progress in spoken dialogue systems, there is a lack of comprehensive surveys that systematically organize and analyze these systems.
  arXiv  Detail & Related papers  (2024-11-15T04:16:45Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Bridging Information Gaps in Dialogues With Grounded Exchanges Using Knowledge Graphs [4.449835214520727]
  We study the potential of large language models for conversational grounding. Our approach involves annotating human conversations across five knowledge domains to create a new dialogue corpus called BridgeKG. Our findings offer insights into how these models use in-context learning for conversational grounding tasks and common prediction errors.
  arXiv  Detail & Related papers  (2024-08-02T08:07:15Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)
• Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues [8.652711997920463]
  Existing works have improved the performance of dialogue systems by intentionally learning interlocutor personas with sophisticated network structures. This study proposes a method of learning to memorize entailment and discourse relations for persona-consistent dialogue tasks.
  arXiv  Detail & Related papers  (2023-01-12T08:37:00Z)
• CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning [85.3987745097806]
  offline reinforcement learning can be used to train dialogue agents entirely using static datasets collected from human speakers. Experiments show that recently developed offline RL methods can be combined with language models to yield realistic dialogue agents.
  arXiv  Detail & Related papers  (2022-04-18T17:43:21Z)
• "How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations [87.95711406978157]
  This work presents a new benchmark on spoken task-oriented conversations. We study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2021-09-28T04:51:04Z)
• Structural Pre-training for Dialogue Comprehension [51.215629336320305]
  We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
  arXiv  Detail & Related papers  (2021-05-23T15:16:54Z)
• Advances in Multi-turn Dialogue Comprehension: A Survey [51.215629336320305]
  We review the previous methods from the perspective of dialogue modeling. We discuss three typical patterns of dialogue modeling that are widely-used in dialogue comprehension tasks.
  arXiv  Detail & Related papers  (2021-03-04T15:50:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.