Predicting Cognitive Load from Speech and Interaction Dynamics in Dyadic Conversations

• URL: http://arxiv.org/abs/2606.12971v1
• Date: Thu, 11 Jun 2026 07:05:26 GMT
• Title: Predicting Cognitive Load from Speech and Interaction Dynamics in Dyadic Conversations
• Abstract summary: Estimating cognitive load from speech has largely been studied in controlled laboratory settings.<n>We analyze audio from 53 dyads performing nine collaborative tasks and extract static acoustic, dynamic, and interaction features to train a two-head Gated Recurrent Unit encoder.<n>Results show conversational interaction provides useful signals for predicting cognitive load related to time pressure, mental work, effort, and task performance.
• Score: 5.53329677986653
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating cognitive load from speech has largely been studied in controlled laboratory settings, with limited understanding of its reliability in natural collaborative conversations. We investigate whether speech and interaction dynamics predict perceived cognitive load during dyadic conversations. We analyze audio from 53 dyads performing nine collaborative tasks and extract static acoustic, dynamic, and interaction features to train a two-head Gated Recurrent Unit encoder to predict cognitive load scores. Results show conversational interaction provides useful signals for predicting cognitive load related to time pressure, mental work, effort, and task performance. Temporal demand is associated with turn-taking dynamics such as overlap and speaker switch, while mental demand is linked to imbalanced participation between speakers. These findings highlight the importance of task structure and conversational interaction for modeling cognitive load in natural collaborative settings.

Related papers

• Shifting Relational Paradigms for Affective Computing: Affective Resonance, Vitality Affects, and Vocal Interaction Fields [5.187859532730832]
  Affective computing has largely followed an individual-state paradigm.<n>We argue this framing is incomplete for interaction.<n>We propose a relational framework in which the primary unit of affective analysis is the interactional field constituted within vocal dynamics.
  arXiv  Detail & Related papers  (2026-09-09T08:16:32Z)
• Predicting Turn-Taking Outcomes in Multi-Party Conversation: Interpretable Modelling of Speech and Gaze Dynamics with Interpersonal Closeness [1.302669708492884]
  This study models how gaze and speech, together with perceived interpersonal closeness, signal conversational floor changes in free four-person dialogue.<n>Using the GaMMA corpus, we trained logistic regression models using interpretable, behaviourally motivated features extracted before each turn-taking event.<n>Results show that gaze features capture predictive structure, and that combining them with loudness improves performance.
  arXiv  Detail & Related papers  (2026-08-28T06:57:54Z)
• TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech [15.290622671304305]
  We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions.<n>We propose TRACE, a window-level framework that models dyadic interaction as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations.
  arXiv  Detail & Related papers  (2026-06-29T16:42:24Z)
• The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning [76.96796481909581]
  Internal cognitive processing is instrumental in formulating high-quality responses.<n>We propose a novel method named FLAIR that conducts latent thinking simultaneously with speech perception.<n>Our approach aligns seamlessly with spoken dialogue systems.
  arXiv  Detail & Related papers  (2026-03-18T15:30:29Z)
• Acoustic and Facial Markers of Perceived Conversational Success in Spontaneous Speech [0.0]
  We analyze a large corpus of spontaneous dyadic Zoom conversations to examine how conversational dynamics relate to perceived interaction quality.<n>Results demonstrate that entrainment reliably detected in spontaneous speech and correlates with higher perceived success.
  arXiv  Detail & Related papers  (2026-03-03T19:11:38Z)
• Hear You in Silence: Designing for Active Listening in Human Interaction with Conversational Agents Using Context-Aware Pacing [17.874659591744486]
  "Active listening" is overlooked in the design of Conversational Agents (CAs)<n>We distill five context-aware pacing strategies: Reflective Silence, Facilitative Silence, Empathic Silence, Holding Space, and Immediate Response.<n>This work shows how insights from human conversation like context-aware pacing can empower the design of more empathic human-AI communication.
  arXiv  Detail & Related papers  (2026-02-05T19:08:06Z)
• Chronological Thinking in Full-Duplex Spoken Dialogue Language Models [66.84843878538207]
  Chronological Thinking aims to improve response quality in full SDLMs.<n>No additional latency: once the user stops speaking, the agent halts thinking and begins speaking without further delay.<n>Results: Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations.
  arXiv  Detail & Related papers  (2025-10-02T10:28:11Z)
• Relational Dissonance in Human-AI Interactions: The Case of Knowledge Work [4.678160092569168]
  We find that fundamental ontological and relational ambiguities inherent in anthropomorphic conversational agents make it difficult for individuals to maintain consistent relational stances toward them.<n>Our findings indicate that people's articulated positioning toward such agents often differs from the relational dynamics that occur during interactions.<n>We propose the concept of relational dissonance to help researchers, designers, and policymakers recognize the resulting tensions.
  arXiv  Detail & Related papers  (2025-09-19T10:15:55Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• Dyadic Interaction Modeling for Social Behavior Generation [6.626277726145613]
  We present an effective framework for creating 3D facial motions in dyadic interactions. The heart of our framework is Dyadic Interaction Modeling (DIM), a pre-training approach. Experiments demonstrate the superiority of our framework in generating listener motions.
  arXiv  Detail & Related papers  (2024-03-14T03:21:33Z)
• MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation [62.44907105496227]
  MindDial is a novel conversational framework that can generate situated free-form responses with theory-of-mind modeling. We introduce an explicit mind module that can track the speaker's belief and the speaker's prediction of the listener's belief. Our framework is applied to both prompting and fine-tuning-based models, and is evaluated across scenarios involving both common ground alignment and negotiation.
  arXiv  Detail & Related papers  (2023-06-27T07:24:32Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.