Behavioral Engagement in VR-Based Sign Language Learning: Visual Attention as a Predictor of Performance and Temporal Dynamics

• URL: http://arxiv.org/abs/2603.19535v1
• Date: Fri, 20 Mar 2026 00:24:15 GMT
• Title: Behavioral Engagement in VR-Based Sign Language Learning: Visual Attention as a Predictor of Performance and Temporal Dynamics
• Abstract summary: This study analyzes behavioral engagement in SONAR, a virtual reality application designed for sign language training and validation.<n>We focus on three automatically derived engagement indicators (Visual Attention (VA), Video Replay Frequency (VRF), and Post-Playback Viewing Time (PPVT)) and examine their relationship with learning performance.
• Score: 3.1690235522182113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study analyzes behavioral engagement in SONAR, a virtual reality application designed for sign language training and validation. We focus on three automatically derived engagement indicators (Visual Attention (VA), Video Replay Frequency (VRF), and Post-Playback Viewing Time (PPVT)) and examine their relationship with learning performance. Participants completed a self-paced Training phase, followed by a Validation quiz assessing retention. We employed Pearson correlation analysis to examine the relationships between engagement indicators and quiz performance, followed by binomial Generalized Linear Model (GLM) regression to assess their joint predictive contributions. Additionally, we conducted temporal analysis by aggregating moment-to-moment VA traces across all learners to characterize engagement dynamics during the learning session. Results show that VA exhibits a strong positive correlation with quiz performance,followed by PPVT, whereas VRF shows no meaningful association. A binomial GLM confirms that VA and PPVT are significant predictors of learning success, jointly explaining a substantial proportion of performance variance. Going beyond outcome-oriented analysis, we characterize temporal engagement patterns by aggregating moment-to-moment VA traces across all learners. The temporal profile reveals distinct attention peaks aligned with informationally dense segments of both training and validation videos, as well as phase-specific engagement dynamics, including initial acclimatization, oscillatory attention cycles during learning, and pronounced attentional peaks during assessment. Together, these findings highlight the central role of sustained and strategically allocated visual attention in VR-based sign language learning and demonstrate the value of behavioral trace data for understanding and predicting learner engagement in immersive environments.

Related papers

• Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control [10.054153317469096]
  We propose Observation-Grounded Self-Predictive Representations (OG-SPR) for continuous control.<n>OG-SPR incorporates two core auxiliary objectives: multi-step latent self-prediction and next-observation prediction.<n> Experiments show that OG-SPR improves aggregate performance over state-of-the-art self-predictive and observation-predictive RL methods.
  arXiv  Detail & Related papers  (2026-08-06T13:02:59Z)
• A Causality-aware Infer-diagnose-refine Framework for Test-time Modality Adaptation in VLA Models [16.665710235622214]
  We propose an infer-diagnose-refine (IDR) framework to fuse modalities in vision-language-action (VLA) models.<n>IDR first infers actions under factual and counterfactual scenarios of visual observations, and then diagnoses the causal effects of visual observations as the estimated dynamic importance.<n>Experiments on both simulation benchmarks and real-world tasks show improvements in overall performance across multiple VLA backbones.
  arXiv  Detail & Related papers  (2026-07-28T09:56:42Z)
• AI-Assisted Competency Assessment from Egocentric Video in Simulation-Based Nursing Education [9.996771938125582]
  Vision-language models have emerged as a promising tool for understanding complex visual behavior.<n>We extract action timelines from egocentric nursing simulation video using frozen visual encoders and few-shot learning.<n>We observe a negative trend between recognition accuracy and competency.
  arXiv  Detail & Related papers  (2026-05-16T02:07:35Z)
• EduGage: Methods and Dataset for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning [6.377791358687315]
  We develop and evaluate a system for engagement estimation, compare different sensing modalities, and analyze the feasibility and effectiveness of multimodal modeling for characterizing learner engagement.<n>Our results suggest that fine-grained engagement estimation is feasible but inherently noisy, and that practical systems should prioritize lightweight combinations of behavioral and physiological signals over full multimodal instrumentation.
  arXiv  Detail & Related papers  (2026-05-02T04:29:16Z)
• Disentangling Learning from Judgment: Representation Learning for Open Response Analytics [0.0]
  Open-ended responses are central to learning, yet automated scoring often conflates what students wrote with how teachers grade.<n>We present an analytics-first framework that separates content signals from rater tendencies, making judgments visible and auditable via analytics.
  arXiv  Detail & Related papers  (2025-12-30T02:06:28Z)
• Unleashing Perception-Time Scaling to Multimodal Reasoning Models [60.578179197783754]
  Recent advances in inference-time scaling have substantially enhanced the reasoning capabilities of Large Vision-Language Models (LVLMs)<n>Inspired by this success, similar strategies have been applied to multimodal reasoning, yet their impact on visual perception remains unclear.<n>We propose Perception-Time Scaling (PTS), a novel paradigm that encourages token-rich perception and decomposes complex perception problems into intermediate tractable sub-problems.
  arXiv  Detail & Related papers  (2025-10-10T03:17:52Z)
• Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruning [62.23671919314693]
  Large language models (LLMs) have demonstrated significant improvements in contextual understanding.<n>However, their ability to attend to truly critical information during long-context reasoning and generation still falls behind the pace.<n>We introduce a two-stage framework called Learning to Focus (LeaF) to mitigate confounding factors.
  arXiv  Detail & Related papers  (2025-06-09T15:16:39Z)
• Dynamic Programming Techniques for Enhancing Cognitive Representation in Knowledge Tracing [125.75923987618977]
  We propose the Cognitive Representation Dynamic Programming based Knowledge Tracing (CRDP-KT) model.<n>It is a dynamic programming algorithm to optimize cognitive representations based on the difficulty of the questions and the performance intervals between them.<n>It provides more accurate and systematic input features for subsequent model training, thereby minimizing distortion in the simulation of cognitive states.
  arXiv  Detail & Related papers  (2025-06-03T14:44:48Z)
• Two-Stage Representation Learning for Analyzing Movement Behavior Dynamics in People Living with Dementia [44.39545678576284]
  This study analyzes home activity data from individuals living with dementia by proposing a two-stage, self-supervised learning approach.<n>The first stage converts time-series activities into text sequences encoded by a pre-trained language model.<n>This PageRank vector captures latent state transitions, effectively compressing complex behaviour data into a succinct form.
  arXiv  Detail & Related papers  (2025-02-13T10:57:25Z)
• A General Model for Detecting Learner Engagement: Implementation and Evaluation [0.0]
  This paper proposes a general, lightweight model for selecting and processing features to detect learners' engagement levels. We analyzed the videos from the publicly available DAiSEE dataset to capture the dynamic essence of learner engagement. The suggested model achieves an accuracy of 68.57% in a specific implementation and outperforms the studied state-of-the-art models detecting learners' engagement levels.
  arXiv  Detail & Related papers  (2024-05-07T12:11:15Z)
• Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
  Engagement measurement finds application in healthcare, education, services. Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
  arXiv  Detail & Related papers  (2024-04-05T20:39:16Z)
• An Empirical Study on the Power of Future Prediction in Partially Observable Environments [15.773444560355694]
  Self-predictive auxiliary tasks have been shown to improve performance in fully observed settings, but their role in partial observability remains underexplored.<n>We introduce $textttDRL2$, an approach that explicitly decouples representation learning from reinforcement learning.<n>Our findings reinforce the idea that future prediction performance serves as a reliable indicator of representation quality and contributes to improved RL performance.
  arXiv  Detail & Related papers  (2024-02-11T04:53:40Z)
• Temporal Interest Network for User Response Prediction [33.95334897033641]
  We propose a Temporal Interest Network (TIN) to capture the semantic-temporal correlation simultaneously between behaviors and the target. TIN has been successfully deployed in production since October 2023, serving the WeChat Moments traffic.
  arXiv  Detail & Related papers  (2023-08-15T05:48:44Z)
• Anticipating the Unseen Discrepancy for Vision and Language Navigation [63.399180481818405]
  Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. We propose Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
  arXiv  Detail & Related papers  (2022-09-10T19:04:40Z)
• Self-Regulated Learning for Egocentric Video Activity Anticipation [147.9783215348252]
  Self-Regulated Learning (SRL) aims to regulate the intermediate representation consecutively to produce representation that emphasizes the novel information in the frame of the current time-stamp. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets.
  arXiv  Detail & Related papers  (2021-11-23T03:29:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.