Detecting Cognitive Impairment and Psychological Well-being among Older Adults Using Facial, Acoustic, Linguistic, and Cardiovascular Patterns Derived from Remote Conversations

• URL: http://arxiv.org/abs/2412.14194v3
• Date: Thu, 09 Jan 2025 20:16:41 GMT
• Title: Detecting Cognitive Impairment and Psychological Well-being among Older Adults Using Facial, Acoustic, Linguistic, and Cardiovascular Patterns Derived from Remote Conversations
• Authors: Xiaofan Mu, Salman Seyedi, Iris Zheng, Zifan Jiang, Liu Chen, Bolaji Omofojoye, Rachel Hershenberg, Allan I. Levey, Gari D. Clifford, Hiroko H. Dodge, Hyeokhyen Kwon,
• Abstract summary: The aging society urgently requires scalable methods to monitor cognitive decline and identify social and psychological factors indicative of dementia risk in older adults.<n>Recent advances in machine learning offer new opportunities to remotely detect cognitive impairment and assess associated factors, such as neuroticism and psychological well-being.<n>Our experiment showed that speech and language patterns were more useful for quantifying cognitive impairment, whereas facial expression and cardiovascular patterns were more useful for quantifying personality and psychological well-being.
• Score: 5.923344966877598
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The aging society urgently requires scalable methods to monitor cognitive decline and identify social and psychological factors indicative of dementia risk in older adults. Our machine learning (ML) models captured facial, acoustic, linguistic, and cardiovascular features from 39 individuals with normal cognition or Mild Cognitive Impairment derived from remote video conversations and classified cognitive status, social isolation, neuroticism, and psychological well-being. Our model could distinguish Clinical Dementia Rating Scale (CDR) of 0.5 (vs. 0) with 0.78 area under the receiver operating characteristic curve (AUC), social isolation with 0.75 AUC, neuroticism with 0.71 AUC, and negative affect scales with 0.79 AUC. Recent advances in machine learning offer new opportunities to remotely detect cognitive impairment and assess associated factors, such as neuroticism and psychological well-being. Our experiment showed that speech and language patterns were more useful for quantifying cognitive impairment, whereas facial expression and cardiovascular patterns using photoplethysmography (PPG) were more useful for quantifying personality and psychological well-being.

Related papers

• KidVis: Do Multimodal Large Language Models Possess the Visual Perceptual Capabilities of a 6-Year-Old? [79.27736230305516]
  We introduce KidVis, a novel benchmark grounded in the theory of human visual development.<n> evaluating 20 state-of-the-art MLLMs against a human physiological baseline reveals a stark performance disparity.<n>This study confirms that current MLLMs, despite their reasoning prowess, lack the essential physiological perceptual primitives required for generalized visual intelligence.
  arXiv  Detail & Related papers  (2026-01-13T07:32:50Z)
• Interpretable Machine Learning for Cognitive Aging: Handling Missing Data and Uncovering Social Determinant [28.20784930277189]
  Early detection of Alzheimer's disease (AD) is crucial because its neurodegenerative effects are irreversible.<n>We predict cognitive performance from social determinants of health using the NIH NIA-supported PREPARE Challenge Phase 2 dataset.
  arXiv  Detail & Related papers  (2025-10-13T03:04:10Z)
• Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding [50.34821397821815]
  Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities.<n>Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains.
  arXiv  Detail & Related papers  (2025-10-06T15:16:45Z)
• Deep Learning-Based Detection of Cognitive Impairment from Passive Smartphone Sensing with Routine-Aware Augmentation and Demographic Personalization [9.633221910592804]
  Early detection of cognitive impairment is critical for timely diagnosis and intervention.<n> Passive smartphone sensing has emerged as a promising approach for naturalistic and continuous cognitive monitoring.<n>We implement a Long Short-Term Memory (LSTM) model to detect cognitive impairment from sequences of daily behavioral features.
  arXiv  Detail & Related papers  (2025-09-27T07:08:25Z)
• Emotion Recognition in Older Adults with Quantum Machine Learning and Wearable Sensors [0.5188841610098435]
  We investigate the feasibility of inferring emotional states exclusively from physiological signals.<n>We conduct a performance comparison of classical machine learning algorithms and hybrid quantum machine learning (QML) methods.<n>Our results indicate that the quantum-enhanced SVM surpasses classical counterparts in classification performance across all emotion categories.
  arXiv  Detail & Related papers  (2025-07-10T21:12:12Z)
• Investigating VLM Hallucination from a Cognitive Psychology Perspective: A First Step Toward Interpretation with Intriguing Observations [60.63340688538124]
  Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs)<n>Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations.<n>In this work, we introduce a psychological taxonomy, categorizing VLMs' cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority.
  arXiv  Detail & Related papers  (2025-07-03T19:03:16Z)
• Naturalistic Language-related Movie-Watching fMRI Task for Detecting Neurocognitive Decline and Disorder [60.84344168388442]
  Language-related functional magnetic resonance imaging (fMRI) may be a promising approach for detecting cognitive decline and early NCD.<n>We examined the effectiveness of this task among 97 non-demented Chinese older adults from Hong Kong.<n>The study demonstrated the potential of the naturalistic language-related fMRI task for early detection of aging-related cognitive decline and NCD.
  arXiv  Detail & Related papers  (2025-06-10T16:58:47Z)
• Deep Learning Characterizes Depression and Suicidal Ideation from Eye Movements [17.985259248390943]
  Depression and suicidal ideation, in particular, lack objective biomarkers, with screening and diagnosis primarily relying on self-reports and clinical interviews. We investigated eye tracking as a potential marker modality for screening purposes. We recorded eye-tracking sequences from 126 young adults as they read and responded to affective sentences, and subsequently developed a deep learning framework to predict their clinical status.
  arXiv  Detail & Related papers  (2025-04-29T17:11:13Z)
• Early Detection of Cognitive Impairment in Elderly using a Passive FPVS-EEG BCI and Machine Learning -- Extended Version [1.3499500088995462]
  Early dementia diagnosis requires biomarkers sensitive to both structural and functional brain changes. Current cognitive assessments often rely on behavioral responses, making them susceptible to factors like effort, practice effects, and educational background. This work introduces a novel approach, leveraging a lightweight convolutional neural network (CNN) to infer cognitive impairment levels directly from electroencephalography (EEG) data.
  arXiv  Detail & Related papers  (2025-04-15T08:34:13Z)
• Explainable Brain Age Gap Prediction in Neurodegenerative Conditions using coVariance Neural Networks [94.06526659234756]
  Black-box machine learning approaches to brain age gap prediction have limited practical utility. We apply the VNN-based approach to study brain age gap using cortical thickness features for various prevalent neurodegenerative conditions. Our results reveal distinct anatomic patterns for brain age gap in Alzheimer's disease, frontotemporal dementia, and atypical Parkinsonian disorders.
  arXiv  Detail & Related papers  (2025-01-02T19:37:09Z)
• Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder [53.575426835313536]
  This paper explores language-related functional changes in older NCD adults using LLM-based fMRI encoding and brain scores. We analyze the correlation between brain scores and cognitive scores at both whole-brain and language-related ROI levels. Our findings reveal that higher cognitive abilities correspond to better brain scores, with correlations peaking in the middle temporal gyrus.
  arXiv  Detail & Related papers  (2024-07-15T01:09:08Z)
• Automatic detection of cognitive impairment in elderly people using an entertainment chatbot with Natural Language Processing capabilities [8.032202552952299]
  We present an intelligent conversational system for entertaining elderly people with news of their interest that monitors cognitive impairment transparently. We create dialogue flows automatically from updated news items using Natural Language Generation techniques. The system infers the gold standard of the answers to the questions, so it can assess cognitive capabilities automatically.
  arXiv  Detail & Related papers  (2024-05-28T19:17:48Z)
• PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents [68.50571379012621]
  Psychological measurement is essential for mental health, self-understanding, and personal development. PsychoGAT (Psychological Game AgenTs) achieves statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity.
  arXiv  Detail & Related papers  (2024-02-19T18:00:30Z)
• Assessing cognitive function among older adults using machine learning and wearable device data: a feasibility study [3.0872517448897465]
  We developed prediction models to differentiate older adults with normal cognition from those with poor cognition. Activity and sleep parameters were also more strongly associated with processing speed, working memory, and attention compared to other cognitive fluency.
  arXiv  Detail & Related papers  (2023-08-28T00:07:55Z)
• Deep learning reveals the common spectrum underlying multiple brain disorders in youth and elders from brain functional networks [53.257804915263165]
  Brain disorders in the early and late life of humans potentially share pathological alterations in brain functions. Key evidence from neuroimaging data for pathological commonness remains unrevealed. We build a deep learning model, using multi-site functional magnetic resonance imaging data, for classifying 5 different brain disorders from healthy controls.
  arXiv  Detail & Related papers  (2023-02-23T09:22:05Z)
• Multi-modal deep learning system for depression and anxiety detection [4.539240676216187]
  We propose a multi-modal system for the screening of depression and anxiety from self-administered speech tasks. We find that augmenting hand-crafted features with deep-learned features improves our overall classification F1 score.
  arXiv  Detail & Related papers  (2022-12-30T00:02:58Z)
• Acoustic-Linguistic Features for Modeling Neurological Task Score in Alzheimer's [1.290382979353427]
  Natural language processing and machine learning provide promising techniques for reliably detecting Alzheimer's disease. We compare and contrast the performance of ten linear regression models for predicting Mini-Mental Status exam scores. We find that, for the given task, handcrafted linguistic features are more significant than acoustic and learned features.
  arXiv  Detail & Related papers  (2022-09-13T15:35:31Z)
• Bias Reducing Multitask Learning on Mental Health Prediction [18.32551434711739]
  There has been an increase in research in developing machine learning models for mental health detection or prediction. In this work, we aim to perform a fairness analysis and implement a multi-task learning based bias mitigation method on anxiety prediction models. Our analysis showed that our anxiety prediction base model introduced some bias with regards to age, income, ethnicity, and whether a participant is born in the U.S. or not.
  arXiv  Detail & Related papers  (2022-08-07T02:28:32Z)
• The Face of Affective Disorders [7.4005714204825646]
  We study the statistical properties of facial behaviour altered by the regulation of brain arousal in the clinical domain of psychiatry. We name the presented measurement in the sense of the classical scalp based obtrusive sensors Opto Electronic Encephalography (OEG) which relies solely on modern camera based real-time signal processing and computer vision.
  arXiv  Detail & Related papers  (2022-08-02T11:28:17Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)
• CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals [60.921888445317705]
  We propose a CogAlign approach to integrate cognitive language processing signals into natural language processing models. We show that CogAlign achieves significant improvements with multiple cognitive features over state-of-the-art models on public datasets.
  arXiv  Detail & Related papers  (2021-06-10T07:10:25Z)
• Predicting Early Indicators of Cognitive Decline from Verbal Utterances [2.387625146176821]
  Dementia is a group of irreversible, chronic, and progressive neurodegenerative disorders resulting in impaired memory, communication, and thought processes. We measure the feasibility of using the linguistic characteristics of verbal utterances elicited during neuropsychological exams to distinguish between elderly control groups, people with MCI, people diagnosed with possible Alzheimer's disease (AD), and probable AD. Our experiments show that a combination of contextual and psycholinguistic features extracted by a Support Vector Machine improved distinguishing the verbal utterances of elderly controls, people with MCI, possible AD, and probable AD.
  arXiv  Detail & Related papers  (2020-11-19T02:24:11Z)
• MET: Multimodal Perception of Engagement for Telehealth [52.54282887530756]
  We present MET, a learning-based algorithm for perceiving a human's level of engagement from videos. We release a new dataset, MEDICA, for mental health patient engagement detection.
  arXiv  Detail & Related papers  (2020-11-17T15:18:38Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.