Fairness-Aware Multimodal Transformer Modeling for Real-Time Student Attention Estimation

• URL: http://arxiv.org/abs/2609.02232v1
• Date: Wed, 02 Sep 2026 07:42:19 GMT
• Title: Fairness-Aware Multimodal Transformer Modeling for Real-Time Student Attention Estimation
• Abstract summary: This study evaluates fairness-aware multimodal temporal models on DIPSER.<n> DIPSER is a naturalistic classroom dataset combining facial images, wearable-sensor measurements, attention annotations, and automatically inferred demographic metadata.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated student-attention estimation can support learning analytics, but aggregate predictive metrics can conceal demographic disparities. This study evaluates fairness-aware multimodal temporal models on DIPSER, a naturalistic classroom dataset combining facial images, wearable-sensor measurements, attention annotations, and automatically inferred demographic metadata. Three baselines are compared across 10 training seeds: a Visual GRU, a Sensor GRU, and a Residual Fusion Transformer. The multimodal model achieves the best mean test performance (MAE 0.283, RMSE 0.363) and the lowest worst-group error among the evaluated baselines, although its gain over the Visual GRU is modest. Gender- and age-targeted MAE-gap regularization reduces disparities on validation data, but these gains do not consistently transfer to held-out subjects or repeated subject-level splits. On an NVIDIA A100-SXM4-40GB GPU, the warm end-to-end pipeline averages 50.65 ms per prediction window at a one-second stride, while the temporal model itself requires 1.02 ms. The findings show that multimodal fusion can modestly improve prediction and worst-group performance, but validation-level fairness gains should not be assumed to generalize. Robust fairness assessment therefore requires subgroup-aware evaluation, repeated subject-level validation, and larger, better balanced demographic samples.

Related papers

• Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment [75.88231994139132]
  We show that even state-of-the-art models like GPT-4.1 achieve only 57.4% accuracy in predicting modal preferences.<n>We find significant pre-existing performance biases, where models better emulate young, male, Chinese, and Christian personas.
  arXiv  Detail & Related papers  (2026-04-14T15:06:13Z)
• Enabling Granular Subgroup Level Model Evaluations by Generating Synthetic Medical Time Series [3.5347586732342435]
  We introduce textitEnhanced TimeAutoDiff, which augments the latent diffusion objective with distribution-alignment penalties.<n>We benchmark all models on MIMIC-III and eICU, on 24-hour mortality and binary length-of-stay tasks.<n>For 32 intersectional subgroups, large synthetic cohorts cut-level AUROC estimation error by up to 50%.
  arXiv  Detail & Related papers  (2025-10-22T16:17:29Z)
• Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting [3.1959623025848405]
  Deep neural networks often rely on spurious correlations in training data, leading to biased or unfair predictions in safety-critical domains such as medicine and autonomous driving.<n>Recent advances in machine unlearning provide a promising alternative for post-hoc model correction.
  arXiv  Detail & Related papers  (2025-09-09T07:25:51Z)
• Test-Time Consistency in Vision Language Models [26.475993408532304]
  Vision-Language Models (VLMs) have achieved impressive performance across a wide range of multimodal tasks.<n>Recent benchmarks, such as MM-R3, highlight that even state-of-the-art VLMs can produce divergent predictions across semantically equivalent inputs.<n>We propose a simple and effective test-time consistency framework that enhances semantic consistency without supervised re-training.
  arXiv  Detail & Related papers  (2025-06-27T17:09:44Z)
• Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning [77.120955854093]
  We show that data diversity can be a strong predictor of generalization in language models.<n>We introduce G-Vendi, a metric that quantifies diversity via the entropy of model-induced gradients.<n>We present Prismatic Synthesis, a framework for generating diverse synthetic data.
  arXiv  Detail & Related papers  (2025-05-26T16:05:10Z)
• Performance of Machine Learning Classifiers for Anomaly Detection in Cyber Security Applications [0.1601392577755919]
  This work empirically evaluates machine learning models on two imbalanced public datasets.<n>Models tested include eXtreme Gradient Boosting (XGB) and Multi Layer Perceptron (MLP)<n>IterativeImputer results are comparable to mean and median, but not recommended for large datasets due to increased complexity and execution time.
  arXiv  Detail & Related papers  (2025-04-26T02:43:27Z)
• SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models [74.40683913645731]
  Zero-shot multi-label recognition (MLR) with Vision-Language Models (VLMs) faces significant challenges without training data, model tuning, or architectural modifications.<n>Our work proposes a novel solution treating VLMs as black boxes, leveraging scores without training data or ground truth.<n>Analysis of these prompt scores reveals VLM biases and AND''/OR' signal ambiguities, notably that maximum scores are surprisingly suboptimal compared to second-highest scores.
  arXiv  Detail & Related papers  (2025-02-24T07:15:05Z)
• No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance [68.18779562801762]
  multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance. Our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.
  arXiv  Detail & Related papers  (2024-04-04T17:58:02Z)
• Bias Amplification Enhances Minority Group Performance [10.380812738348899]
  We propose BAM, a novel two-stage training algorithm. In the first stage, the model is trained using a bias amplification scheme via introducing a learnable auxiliary variable for each training sample. In the second stage, we upweight the samples that the bias-amplified model misclassifies, and then continue training the same model on the reweighted dataset.
  arXiv  Detail & Related papers  (2023-09-13T04:40:08Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Local Clustering with Mean Teacher for Semi-supervised Learning [8.54739245813914]
  Mean Teacher (MT) model of Tarvainen and Valpola has shown favorable performance on several semi-supervised benchmark datasets. We propose a simple yet effective method called Local Clustering (LC) to mitigate the effect of confirmation bias.
  arXiv  Detail & Related papers  (2020-04-20T22:31:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.