Passive Dementia Screening via Facial Temporal Micro-Dynamics Analysis of In-the-Wild Talking-Head Video

• URL: http://arxiv.org/abs/2511.13802v2
• Date: Wed, 26 Nov 2025 09:04:03 GMT
• Title: Passive Dementia Screening via Facial Temporal Micro-Dynamics Analysis of In-the-Wild Talking-Head Video
• Abstract summary: We target passive dementia screening from short camera-facing talking head video.<n>We develop a facial temporal micro dynamics analysis for language free detection of early neuro cognitive change.
• Score: 23.99097268182305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We target passive dementia screening from short camera-facing talking head video, developing a facial temporal micro dynamics analysis for language free detection of early neuro cognitive change. This enables unscripted, in the wild video analysis at scale to capture natural facial behaviors, transferrable across devices, topics, and cultures without active intervention by clinicians or researchers during recording. Most existing resources prioritize speech or scripted interviews, limiting use outside clinics and coupling predictions to language and transcription. In contrast, we identify and analyze whether temporal facial kinematics, including blink dynamics, small mouth jaw motions, gaze variability, and subtle head adjustments, are sufficient for dementia screening without speech or text. By stabilizing facial signals, we convert these micro movements into interpretable facial microdynamic time series, smooth them, and summarize short windows into compact clip level statistics for screening. Each window is encoded by its activity mix (the relative share of motion across streams), thus the predictor analyzes the distribution of motion across streams rather than its magnitude, making per channel effects transparent. We also introduce YT DemTalk, a new dataset curated from publicly available, in the wild camera facing videos. It contains 300 clips (150 with self reported dementia, 150 controls) to test our model and offer a first benchmarking of the corpus. On YT DemTalk, ablations identify gaze lability and mouth/jaw dynamics as the most informative cues, and light weighted shallow classifiers could attain a dementia prediction performance of (AUROC) 0.953, 0.961 Average Precision (AP), 0.851 F1-score, and 0.857 accuracy.

Related papers

• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events [52.031070006859544]
  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.<n>We introduce Moment-Video, a benchmark for diagnosing the temporal fidelity of video MLLMs through momentary visual event understanding.
  arXiv  Detail & Related papers  (2026-06-01T17:32:20Z)
• MedHorizon: Towards Long-context Medical Video Understanding in the Wild [78.79695798197447]
  Real clinical review often requires full-procedure video understanding.<n>Existing benchmarks often assume this evidence has already been localized through images, short clips, or pre-segmented videos.<n>We introduce MedHorizon, an in-the-wild benchmark for long-context medical video understanding.
  arXiv  Detail & Related papers  (2026-05-07T16:37:10Z)
• Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology [6.150572757901948]
  This pilot study evaluates the capability of MLLMs for automated recognition of pathological movements in seizure videos.<n>We assessed the zero-shot performance of state-of-the-art MLLMs on 20 ILAE-defined semiological features across 90 clinical seizure recordings.
  arXiv  Detail & Related papers  (2026-05-05T04:14:38Z)
• Digital FAST: An AI-Driven Multimodal Framework for Rapid and Early Stroke Screening [0.7136933021609076]
  This study presents a fast, non-invasive multimodal deep learning framework for automatic binary stroke screening based on data collected during the F.A.S.T. assessment.<n>The proposed approach integrates complementary information from facial expressions, speech signals, and upper-body movements to enhance diagnostic robustness.
  arXiv  Detail & Related papers  (2026-01-17T03:35:39Z)
• Brain-language fusion enables interactive neural readout and in-silico experimentation [0.8805057433368938]
  CorText is a framework that integrates neural activity directly into the latent space of an large language model.<n>It generates accurate image captions and can answer more detailed questions better than controls, while having access to neural data only.<n>These advances mark a shift from passive decoding toward generative, flexible interfaces between brain activity and language.
  arXiv  Detail & Related papers  (2025-09-28T15:35:25Z)
• HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis [90.74616208952791]
  HM-Talker is a novel framework for generating high-fidelity, temporally coherent talking heads.<n>Explicit cues use Action Units (AUs), anatomically defined facial muscle movements, alongside implicit features to minimize phoneme-viseme misalignment.
  arXiv  Detail & Related papers  (2025-08-14T12:01:52Z)
• SIM: Surface-based fMRI Analysis for Inter-Subject Multimodal Decoding from Movie-Watching Experiments [9.786770726122436]
  Current AI frameworks for brain decoding and encoding, typically train and test models within the same datasets.<n>Key obstacle to model generalisation is the degree of variability of inter-subject cortical organisation.<n>In this paper we address this through the use of surface vision transformers, which build a generalisable model of cortical functional dynamics.
  arXiv  Detail & Related papers  (2025-01-27T20:05:17Z)
• Detection of Mild Cognitive Impairment Using Facial Features in Video Conversations [4.229544696616341]
  Early detection of Mild Cognitive Impairment (MCI) leads to early interventions to slow the progression from MCI into dementia. Deep Learning (DL) algorithms could help achieve early non-invasive, low-cost detection of MCI. This paper presents the detection of MCI in older adults using DL models based only on facial features extracted from video-recorded conversations at home.
  arXiv  Detail & Related papers  (2023-08-29T20:45:41Z)
• Probing Brain Context-Sensitivity with Masked-Attention Generation [87.31930367845125]
  We use GPT-2 transformers to generate word embeddings that capture a fixed amount of contextual information. We then tested whether these embeddings could predict fMRI brain activity in humans listening to naturalistic text.
  arXiv  Detail & Related papers  (2023-05-23T09:36:21Z)
• MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videos [44.72781467904852]
  This paper proposes a novel Multi-branch-Video Vision Transformer (MCViViT) model to distinguish from those with normal cognition by analyzing facial features. The data comes from the I-CONECT, a behavioral intervention trial aimed at improving cognitive function by providing frequent video chats. Our experimental results on I-CONECT dataset show the great potential of MC-ViViT in predicting MCI with a high accuracy of 90.63%.
  arXiv  Detail & Related papers  (2023-04-11T15:42:20Z)
• Decoding speech perception from non-invasive brain recordings [48.46819575538446]
  We introduce a model trained with contrastive-learning to decode self-supervised representations of perceived speech from non-invasive recordings. Our model can identify, from 3 seconds of MEG signals, the corresponding speech segment with up to 41% accuracy out of more than 1,000 distinct possibilities.
  arXiv  Detail & Related papers  (2022-08-25T10:01:43Z)
• DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering [69.9557427451339]
  We propose a framework based on neural radiance field to pursue high-fidelity talking head generation. Specifically, neural radiance field takes lip movements features and personalized attributes as two disentangled conditions. We show that our method achieves significantly better results than state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-03T18:23:38Z)
• Transfer Learning of Deep Spatiotemporal Networks to Model Arbitrarily Long Videos of Seizures [58.720142291102135]
  Detailed analysis of seizure semiology is critical for management of epilepsy patients. We present GESTURES, a novel architecture combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs) We show that an STCNN trained on a HAR dataset can be used in combination with an RNN to accurately represent arbitrarily long videos of seizures.
  arXiv  Detail & Related papers  (2021-06-22T18:40:31Z)
• Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection [118.37239586697139]
  LipForensics is a detection approach capable of both generalising manipulations and withstanding various distortions. It consists in first pretraining a-temporal network to perform visual speech recognition (lipreading) A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without over-fitting to low-level, manipulation-specific artefacts.
  arXiv  Detail & Related papers  (2020-12-14T15:53:56Z)
• Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements [2.6540572249827514]
  Speech disorders such as stuttering disrupt the normal fluency of speech by involuntary repetitions, prolongations and blocking of sounds and syllables. Recent studies have explored automatic detection of stuttering using Artificial Intelligence (AI) based algorithm from respiratory rate, audio, etc. during speech utterance. We hypothesize that pre-speech facial activity in AWS, which can be captured non-invasively, contains enough information to accurately classify the upcoming utterance as either fluent or stuttered.
  arXiv  Detail & Related papers  (2020-10-02T23:45:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.