Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application

• URL: http://arxiv.org/abs/2105.07949v1
• Date: Thu, 29 Apr 2021 20:45:02 GMT
• Title: Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application
• Authors: Abhijit Suresh, Jennifer Jacobs, Vivian Lai, Chenhao Tan, Wayne Ward, James H. Martin, Tamara Sumner
• Abstract summary: We describe the TalkMoves application's cloud-based infrastructure for managing and processing classroom recordings. We discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms.
• Score: 14.851607363136978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as "talk moves" within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application's cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms.

Related papers

• Intelligent Interface: Enhancing Lecture Engagement with Didactic Activity Summaries [0.054204929130712134]
  The prototype utilizes machine learning-based techniques to recognise selected didactic and behavioural teachers' features within lecture video recordings. The system offers flexibility for (future) integration of new/additional machine-learning models and software modules for image and video analysis.
  arXiv  Detail & Related papers  (2024-06-20T12:45:23Z)
• Multimodal Classification of Teaching Activities from University Lecture Recordings [0.9790236766474201]
  We present a multimodal classification algorithm that identifies the type of activity that is being carried out at any time of the lesson. Some academic activities are more easily identifiable with the audio signal while resorting to the text transcription is needed to identify others.
  arXiv  Detail & Related papers  (2023-12-24T08:33:30Z)
• User Adaptive Language Learning Chatbots with a Curriculum [55.63893493019025]
  We adapt lexically constrained decoding to a dialog system, which urges the dialog system to include curriculum-aligned words and phrases in its generated utterances. The evaluation result demonstrates that the dialog system with curriculum infusion improves students' understanding of target words and increases their interest in practicing English.
  arXiv  Detail & Related papers  (2023-04-11T20:41:41Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)
• Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension [81.47133615169203]
  We propose compositional learning for holistic interaction across utterances beyond the sequential contextualization from PrLMs. We employ domain-adaptive training strategies to help the model adapt to the dialogue domains. Experimental results show that our method substantially boosts the strong PrLM baselines in four public benchmark datasets.
  arXiv  Detail & Related papers  (2023-01-10T13:18:25Z)
• The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves [8.090330715662962]
  This paper describes the TalkMoves dataset, composed of 567 human-annotated K-12 mathematics lesson transcripts. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms.
  arXiv  Detail & Related papers  (2022-04-06T18:12:30Z)
• Improving mathematical questioning in teacher training [1.794107419334178]
  High-fidelity, AI-based simulated classroom systems enable teachers to rehearse effective teaching strategies. This paper builds a text-based interactive conversational agent to help teachers practice mathematical questioning skills.
  arXiv  Detail & Related papers  (2021-12-02T05:33:03Z)
• What Would a Teacher Do? Predicting Future Talk Moves [19.952531500315757]
  We introduce a new task, called future talk move prediction (FTMP) It consists of predicting the next talk move given a conversation history with its corresponding talk moves. We introduce a neural network model for this task, which outperforms multiple baselines by a large margin.
  arXiv  Detail & Related papers  (2021-06-09T17:45:16Z)
• Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language [148.0843278195794]
  We propose a new model architecture for learning multi-modal neuro-symbolic representations for video captioning. Our approach uses a dictionary learning-based method of learning relations between videos and their paired text descriptions.
  arXiv  Detail & Related papers  (2020-11-18T20:21:19Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)
• Neural Multi-Task Learning for Teacher Question Detection in Online Classrooms [50.19997675066203]
  We build an end-to-end neural framework that automatically detects questions from teachers' audio recordings. By incorporating multi-task learning techniques, we are able to strengthen the understanding of semantic relations among different types of questions.
  arXiv  Detail & Related papers  (2020-05-16T02:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.