Towards an optimised evaluation of teachers' discourse: The case of engaging messages

• URL: http://arxiv.org/abs/2412.14011v2
• Date: Thu, 19 Dec 2024 08:27:54 GMT
• Title: Towards an optimised evaluation of teachers' discourse: The case of engaging messages
• Abstract summary: This study introduces a new methodology for optimising the assessment of teacher discourse.<n>The first study involved training two large language models on real-world examples from audio-recorded lessons over two academic years.<n>The second study applied these models to transcripts of audio-recorded lessons from a third academic year to examine the frequency and distribution of message types.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating teachers' skills is crucial for enhancing education quality and student outcomes. Teacher discourse, significantly influencing student performance, is a key component. However, coding this discourse can be laborious. This study addresses this issue by introducing a new methodology for optimising the assessment of teacher discourse. The research consisted of two studies, both within the framework of engaging messages used by secondary education teachers. The first study involved training two large language models on real-world examples from audio-recorded lessons over two academic years to identify and classify the engaging messages from the lessons' transcripts. This resulted in sensitivities of 84.31% and 91.11%, and specificities of 97.69% and 86.36% in identification and classification, respectively. The second study applied these models to transcripts of audio-recorded lessons from a third academic year to examine the frequency and distribution of message types by educational level and moment of the academic year. Results showed teachers predominantly use messages emphasising engagement benefits, linked to improved outcomes, while one-third highlighted non-engagement disadvantages, associated with increased anxiety. The use of engaging messages declined in Grade 12 and towards the academic year's end. These findings suggest potential interventions to optimise engaging message use, enhancing teaching quality and student outcomes.

Related papers

• Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons [0.8599595236399581]
  This study examined how learner characteristics affect motivation, learning outcomes, and overall evaluation in three types of dialogue-based lessons.<n>Three types of lessons were generated using a large language model (LLM) and Text-to-Speech (TTS) technology.
  arXiv  Detail & Related papers  (2026-08-21T07:43:50Z)
• Students' Perceptions of Peer Grading [0.8941624592392746]
  We conducted a mixed-source thematic analysis of literature and student discussions on Reddit.<n>The study synthesized evidence from 107 papers, 114 Reddit posts, and 300 comments.<n>Students view peer grading as both beneficial and problematic.
  arXiv  Detail & Related papers  (2026-06-13T02:46:09Z)
• Teaching Audio Models to Reason: A Unified Framework for Source- and Layer-wise Distillation [52.537908557508324]
  We propose a unified knowledge distillation framework to transfer reasoning capabilities from a high-capacity textual model to a student audio model.<n>Our method introduces two key dimensions: source-wise distillation and layer-wise distillation.<n> Experimental results show significant improvements in audio reasoning performance.
  arXiv  Detail & Related papers  (2025-09-23T02:58:16Z)
• In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis [52.42612945266194]
  We propose a new task: generating nuanced, expressive, and time-aware impact summaries.<n>We show that these summaries capture both praise (confirmation citations) and critique (correction citations) through the evolution of fine-grained citation intents.
  arXiv  Detail & Related papers  (2025-05-20T19:11:06Z)
• Semi-automated analysis of audio-recorded lessons: The case of teachers' engaging messages [0.0]
  We collected 2,477 audio-recorded lessons from 75 teachers over two academic years.<n>Using automatic transcription and keyword-based filtering analysis, we identified and classified engaging messages.<n>The most used messages emphasised the future benefits of participating in school activities.
  arXiv  Detail & Related papers  (2024-12-16T18:35:58Z)
• Speechworthy Instruction-tuned Language Models [71.8586707840169]
  We show that both prompting and preference learning increase the speech-suitability of popular instruction-tuned LLMs. We share lexical, syntactical, and qualitative analyses to showcase how each method contributes to improving the speech-suitability of generated responses.
  arXiv  Detail & Related papers  (2024-09-23T02:34:42Z)
• Representational Alignment Supports Effective Machine Teaching [81.19197059407121]
  GRADE is a new controlled experimental setting to study pedagogy and representational alignment. We find that improved representational alignment with a student improves student learning outcomes. However, this effect is moderated by the size and representational diversity of the class being taught.
  arXiv  Detail & Related papers  (2024-06-06T17:48:24Z)
• The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education [3.967610895056427]
  This paper presents the first study that leverages Natural Language Processing (NLP) techniques to assess multiple high-inference instructional practices. We confront two challenges inherent in NLP-based instructional analysis, including noisy and long input data and highly skewed distributions of human ratings.
  arXiv  Detail & Related papers  (2024-04-03T04:15:29Z)
• Does Starting Deep Learning Homework Earlier Improve Grades? [63.20583929886827]
  Students who start a homework assignment earlier and spend more time on it should receive better grades on the assignment. Existing literature on the impact of time spent on homework is not clear-cut and comes mostly from K-12 education. We develop a hierarchical Bayesian model to help make principled conclusions about the impact on student success.
  arXiv  Detail & Related papers  (2023-09-30T09:34:30Z)
• Improving Students With Rubric-Based Self-Assessment and Oral Feedback [2.808134646037882]
  rubrics and oral feedback are approaches to help students improve performance and meet learning outcomes. This paper evaluates the effect of rubrics and oral feedback on student learning outcomes.
  arXiv  Detail & Related papers  (2023-07-24T14:48:28Z)
• Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization [84.86241161706911]
  We show that teacher LLMs can indeed intervene on student reasoning to improve their performance. We also demonstrate that in multi-turn interactions, teacher explanations generalize and learn from explained data. We verify that misaligned teachers can lower student performance to random chance by intentionally misleading them.
  arXiv  Detail & Related papers  (2023-06-15T17:27:20Z)
• Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation [70.92135839545314]
  We propose the dynamic prior knowledge (DPK), which integrates part of teacher's features as the prior knowledge before the feature distillation. Our DPK makes the performance of the student model positively correlated with that of the teacher model, which means that we can further boost the accuracy of students by applying larger teachers.
  arXiv  Detail & Related papers  (2022-06-13T11:52:13Z)
• Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions [19.80258498803113]
  In education, teachers' uptake of student contributions has been linked to higher student achievement. We propose a framework for measuring uptake, by releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake by experts. We find that although repetition captures a significant part of uptake, pJSD outperforms repetition-based baselines, as it is capable of identifying a wider range of uptake phenomena like question answering and reformulation.
  arXiv  Detail & Related papers  (2021-06-07T18:00:06Z)
• Effects of the COVID-19 Pandemic on Learning and Teaching: a Case Study from Higher Education [0.0]
  In December 2019, the first case of SARS-CoV-2 infection was identified in Wuhan, China. This study describes and analyzes the impact of the pandemic on the study patterns of higher education students.
  arXiv  Detail & Related papers  (2021-05-04T11:39:45Z)
• Neural Multi-Task Learning for Teacher Question Detection in Online Classrooms [50.19997675066203]
  We build an end-to-end neural framework that automatically detects questions from teachers' audio recordings. By incorporating multi-task learning techniques, we are able to strengthen the understanding of semantic relations among different types of questions.
  arXiv  Detail & Related papers  (2020-05-16T02:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.