MET: Multimodal Perception of Engagement for Telehealth

• URL: http://arxiv.org/abs/2011.08690v3
• Date: Mon, 22 Aug 2022 21:14:40 GMT
• Title: MET: Multimodal Perception of Engagement for Telehealth
• Authors: Pooja Guhan and Naman Awasthi and and Kathryn McDonald and Kristin Bussell and Dinesh Manocha and Gloria Reeves and Aniket Bera
• Abstract summary: We present MET, a learning-based algorithm for perceiving a human's level of engagement from videos. We release a new dataset, MEDICA, for mental health patient engagement detection.
• Score: 52.54282887530756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present MET, a learning-based algorithm for perceiving a human's level of engagement from videos that give us access to only the face, speech and text. We leverage latent vectors corresponding to Affective and Cognitive features frequently used in psychology literature to understand a person's level of engagement in a semi-supervised GAN-based framework. The method is extremely useful in the case of telehealth. We showcase the efficacy of this method from the perspective of mental health and more specifically how this can be leveraged for a better understanding of patient engagement during telemental health sessions. We also explore the usefulness of our framework and contrast it against existing works in being able to estimate another important mental health indicator, namely valence, and arousal. Our framework reports 40% improvements in RMSE over SOTA method in Engagement Regression and 50% improvements in RMSE over SOTA method in Valence-Arousal Regression. To tackle the scarcity of publicly available datasets in the telemental health space, we release a new dataset, MEDICA, for mental health patient engagement detection. Our dataset, MEDICA consists of 1299 videos, each 3 seconds long. To the best of our knowledge, our approach is the first method capable to model telemental health session data based on psychology-driven Affective and Cognitive features, which also accounts for data sparsity by leveraging a semi-supervised setup. To assert the usefulness of our method, we will also compare the association of the engagement values obtained from our model with the other engagement measures used by psychotherapists.

Related papers

• SouLLMate: An Application Enhancing Diverse Mental Health Support with Adaptive LLMs, Prompt Engineering, and RAG Techniques [9.146311285410631]
  Mental health issues significantly impact individuals' daily lives, yet many do not receive the help they need even with available online resources. This study aims to provide diverse, accessible, stigma-free, personalized, and real-time mental health support through cutting-edge AI technologies.
  arXiv  Detail & Related papers  (2024-10-17T22:04:32Z)
• Neuron-based Personality Trait Induction in Large Language Models [115.08894603023712]
  Large language models (LLMs) have become increasingly proficient at simulating various personality traits. We present a neuron-based approach for personality trait induction in LLMs.
  arXiv  Detail & Related papers  (2024-10-16T07:47:45Z)
• SouLLMate: An Adaptive LLM-Driven System for Advanced Mental Health Support and Assessment, Based on a Systematic Application Survey [9.146311285410631]
  Mental health issues significantly impact individuals' daily lives, yet many do not receive the help they need even with available online resources. This study aims to provide accessible, stigma-free, personalized, and real-time mental health support through cutting-edge AI technologies.
  arXiv  Detail & Related papers  (2024-10-06T17:11:29Z)
• Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
  Engagement measurement finds application in healthcare, education, services. Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
  arXiv  Detail & Related papers  (2024-04-05T20:39:16Z)
• Differential Private Federated Transfer Learning for Mental Health Monitoring in Everyday Settings: A Case Study on Stress Detection [4.439102809224707]
  Mental health conditions necessitate efficient monitoring to mitigate their adverse impacts on life quality. Existing approaches struggle with vulnerabilities to certain cyber-attacks and data insufficiency in real-world applications. We introduce a differential private federated transfer learning framework for mental health monitoring to enhance data privacy and enrich data sufficiency.
  arXiv  Detail & Related papers  (2024-02-16T18:00:04Z)
• From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models [21.427976533706737]
  We take a novel approach that leverages large language models to synthesize clinically useful insights from multi-sensor data. We develop chain of thought prompting methods that use LLMs to generate reasoning about how trends in data relate to conditions like depression and anxiety. We find models like GPT-4 correctly reference numerical data 75% of the time, and clinician participants express strong interest in using this approach to interpret self-tracking data.
  arXiv  Detail & Related papers  (2023-11-21T23:53:27Z)
• A Self-supervised Framework for Improved Data-Driven Monitoring of Stress via Multi-modal Passive Sensing [7.084068935028644]
  We propose a multi-modal semi-supervised framework for tracking physiological precursors of the stress response. Our methodology enables utilizing multi-modal data of differing domains and resolutions from wearable devices. We perform training experiments using a corpus of real-world data on perceived stress.
  arXiv  Detail & Related papers  (2023-03-24T20:34:46Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Learning-based Computer-aided Prescription Model for Parkinson's Disease: A Data-driven Perspective [61.70045118068213]
  We build a dataset by collecting symptoms of PD patients, and their prescription drug provided by neurologists. Then, we build a novel computer-aided prescription model by learning the relation between observed symptoms and prescription drug. For the new coming patients, we could recommend (predict) suitable prescription drug on their observed symptoms by our prescription model.
  arXiv  Detail & Related papers  (2020-07-31T14:34:35Z)
• Detecting Parkinsonian Tremor from IMU Data Collected In-The-Wild using Deep Multiple-Instance Learning [59.74684475991192]
  Parkinson's Disease (PD) is a slowly evolving neuro-logical disease that affects about 1% of the population above 60 years old. PD symptoms include tremor, rigidity and braykinesia. We present a method for automatically identifying tremorous episodes related to PD, based on IMU signals captured via a smartphone device.
  arXiv  Detail & Related papers  (2020-05-06T09:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.