MathBuddy: A Multimodal System for Affective Math Tutoring

• URL: http://arxiv.org/abs/2508.19993v2
• Date: Wed, 24 Sep 2025 19:05:55 GMT
• Title: MathBuddy: A Multimodal System for Affective Math Tutoring
• Authors: Debanjana Kar, Leopold Böss, Dacia Braca, Sebastian Maximilian Dennerlein, Nina Christine Hubig, Philipp Wintersberger, Yufang Hou,
• Abstract summary: MathBuddy is an emotionally aware LLM-powered Math Tutor.<n>It maps the student's emotions to relevant pedagogical strategies, making the tutor-student conversation a more empathetic one.<n>We report a massive 23 point performance gain using the win rate and a 3 point gain at an overall level using DAMR scores.
• Score: 10.968012903118975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid adoption of LLM-based conversational systems is already transforming the landscape of educational technology. However, the current state-of-the-art learning models do not take into account the student's affective states. Multiple studies in educational psychology support the claim that positive or negative emotional states can impact a student's learning capabilities. To bridge this gap, we present MathBuddy, an emotionally aware LLM-powered Math Tutor, which dynamically models the student's emotions and maps them to relevant pedagogical strategies, making the tutor-student conversation a more empathetic one. The student's emotions are captured from the conversational text as well as from their facial expressions. The student's emotions are aggregated from both modalities to confidently prompt our LLM Tutor for an emotionally-aware response. We have evaluated our model using automatic evaluation metrics across eight pedagogical dimensions and user studies. We report a massive 23 point performance gain using the win rate and a 3 point gain at an overall level using DAMR scores which strongly supports our hypothesis of improving LLM-based tutor's pedagogical abilities by modeling students' emotions. Our dataset and code are available at: https://github.com/ITU-NLP/MathBuddy .

Related papers

• PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors [66.56586559631516]
  Large language models (LLMs) have potential as educational tutors.<n>But different tutoring strategies benefit different student personalities.<n>Despite this, current LLM tutoring systems do not take into account student personality traits.
  arXiv  Detail & Related papers  (2026-01-13T10:17:26Z)
• UCO: A Multi-Turn Interactive Reinforcement Learning Method for Adaptive Teaching with Large Language Models [59.693733170193944]
  Large language models (LLMs) are shifting from answer providers to intelligent tutors in educational settings.<n>Recent reinforcement learning approaches address this limitation but face two critical challenges.<n>We propose the Unidirectional Cognitive Optimization (UCO) method to address these challenges.
  arXiv  Detail & Related papers  (2025-11-12T01:27:02Z)
• AI with Emotions: Exploring Emotional Expressions in Large Language Models [0.0]
  Large Language Models (LLMs) play role-play as agents answering questions with specified emotional states.<n>Russell's Circumplex model characterizes emotions along the sleepy-activated (arousal) and pleasure-displeasure (valence) axes.<n> evaluation showed that the emotional states of the generated answers were consistent with the specifications.
  arXiv  Detail & Related papers  (2025-04-20T18:49:25Z)
• Recognizing Emotion Regulation Strategies from Human Behavior with Large Language Models [44.015651538470856]
  Human emotions are often not expressed directly, but regulated according to internal processes and social display rules. No method to automatically classify different emotion regulation strategies in a cross-user scenario exists. We make use of the recently introduced textscDeep corpus for modeling the social display of the emotion shame. A fine-tuned Llama2-7B model is able to classify the utilized emotion regulation strategy with high accuracy.
  arXiv  Detail & Related papers  (2024-08-08T12:47:10Z)
• Toward In-Context Teaching: Adapting Examples to Students' Misconceptions [54.82965010592045]
  We introduce a suite of models and evaluation methods we call AdapT. AToM is a new probabilistic model for adaptive teaching that jointly infers students' past beliefs and optimize for the correctness of future beliefs. Our results highlight both the difficulty of the adaptive teaching task and the potential of learned adaptive models for solving it.
  arXiv  Detail & Related papers  (2024-05-07T17:05:27Z)
• Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling [50.99252242917458]
  Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity. Our model outperforms the baseline models in understanding and rendering emotions.
  arXiv  Detail & Related papers  (2023-12-19T08:47:50Z)
• Enhancing Student Engagement in Online Learning through Facial Expression Analysis and Complex Emotion Recognition using Deep Learning [1.3812010983144802]
  This paper introduces a novel approach employing deep learning techniques based on facial expressions to assess students engagement levels during online learning sessions. To address this challenge, proposed a generation of four complex emotions such as confusion, satisfaction, disappointment, and frustration by combining the basic emotions. The proposed work utilized a Convolutional Neural Network (CNN) model to categorize the fundamental emotional states of learners accurately.
  arXiv  Detail & Related papers  (2023-11-17T06:07:54Z)
• Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench [83.41621219298489]
  We evaluate Large Language Models' (LLMs) anthropomorphic capabilities using the emotion appraisal theory from psychology. We collect a dataset containing over 400 situations that have proven effective in eliciting the eight emotions central to our study. We conduct a human evaluation involving more than 1,200 subjects worldwide.
  arXiv  Detail & Related papers  (2023-08-07T15:18:30Z)
• Large Language Models Understand and Can be Enhanced by Emotional Stimuli [53.53886609012119]
  We take the first step towards exploring the ability of Large Language Models to understand emotional stimuli. Our experiments show that LLMs have a grasp of emotional intelligence, and their performance can be improved with emotional prompts. Our human study results demonstrate that EmotionPrompt significantly boosts the performance of generative tasks.
  arXiv  Detail & Related papers  (2023-07-14T00:57:12Z)
• Enhancing Cognitive Models of Emotions with Representation Learning [58.2386408470585]
  We present a novel deep learning-based framework to generate embedding representations of fine-grained emotions. Our framework integrates a contextualized embedding encoder with a multi-head probing model. Our model is evaluated on the Empathetic Dialogue dataset and shows the state-of-the-art result for classifying 32 emotions.
  arXiv  Detail & Related papers  (2021-04-20T16:55:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.