A Novel Audio-Visual Information Fusion System for Mental Disorders Detection

• URL: http://arxiv.org/abs/2409.02243v1
• Date: Tue, 3 Sep 2024 19:16:36 GMT
• Title: A Novel Audio-Visual Information Fusion System for Mental Disorders Detection
• Authors: Yichun Li, Shuanglin Li, Syed Mohsen Naqvi,
• Abstract summary: Mental disorders are among the foremost contributors to the global healthcare challenge. In this paper, we focus on the emotional expression features of mental disorders and introduce a multimodal mental disorder diagnosis system based on audio-visual information input. Our proposed system is based on spatial-temporal attention networks and innovative uses a less computationally intensive pre-train audio recognition network to fine-tune the video recognition module for better results.
• Score: 6.3344832182228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mental disorders are among the foremost contributors to the global healthcare challenge. Research indicates that timely diagnosis and intervention are vital in treating various mental disorders. However, the early somatization symptoms of certain mental disorders may not be immediately evident, often resulting in their oversight and misdiagnosis. Additionally, the traditional diagnosis methods incur high time and cost. Deep learning methods based on fMRI and EEG have improved the efficiency of the mental disorder detection process. However, the cost of the equipment and trained staff are generally huge. Moreover, most systems are only trained for a specific mental disorder and are not general-purpose. Recently, physiological studies have shown that there are some speech and facial-related symptoms in a few mental disorders (e.g., depression and ADHD). In this paper, we focus on the emotional expression features of mental disorders and introduce a multimodal mental disorder diagnosis system based on audio-visual information input. Our proposed system is based on spatial-temporal attention networks and innovative uses a less computationally intensive pre-train audio recognition network to fine-tune the video recognition module for better results. We also apply the unified system for multiple mental disorders (ADHD and depression) for the first time. The proposed system achieves over 80\% accuracy on the real multimodal ADHD dataset and achieves state-of-the-art results on the depression dataset AVEC 2014.

Related papers

• MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders [59.515827458631975]
  Mental health disorders are one of the most serious diseases in the world. Privacy concerns limit the accessibility of personalized treatment data. MentalArena is a self-play framework to train language models.
  arXiv  Detail & Related papers  (2024-10-09T13:06:40Z)
• Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting [82.64015366154884]
  We study the task of cognitive distortion detection and propose the Diagnosis of Thought (DoT) prompting. DoT performs diagnosis on the patient's speech via three stages: subjectivity assessment to separate the facts and the thoughts; contrastive reasoning to elicit the reasoning processes supporting and contradicting the thoughts; and schema analysis to summarize the cognition schemas. Experiments demonstrate that DoT obtains significant improvements over ChatGPT for cognitive distortion detection, while generating high-quality rationales approved by human experts.
  arXiv  Detail & Related papers  (2023-10-11T02:47:21Z)
• UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training [66.16134293168535]
  We propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics.
  arXiv  Detail & Related papers  (2023-09-13T09:22:49Z)
• Deep learning reveals the common spectrum underlying multiple brain disorders in youth and elders from brain functional networks [53.257804915263165]
  Brain disorders in the early and late life of humans potentially share pathological alterations in brain functions. Key evidence from neuroimaging data for pathological commonness remains unrevealed. We build a deep learning model, using multi-site functional magnetic resonance imaging data, for classifying 5 different brain disorders from healthy controls.
  arXiv  Detail & Related papers  (2023-02-23T09:22:05Z)
• Multi-site Diagnostic Classification Of Schizophrenia Using 3D CNN On Aggregated Task-based fMRI Data [0.0]
  The mechanisms that underlie the development of schizophrenia, as well as its relapse, symptomatology, and treatment, continue to be a mystery. The absence of appropriate analytic tools to deal with the variable and complicated nature of schizophrenia may be one of the factors that contribute to the development of this disorder. Deep learning has the potential to become a powerful tool for understanding the mechanisms that are at the root of schizophrenia.
  arXiv  Detail & Related papers  (2022-10-11T08:12:36Z)
• RobIn: A Robust Interpretable Deep Network for Schizophrenia Diagnosis [12.180396034315807]
  Schizophrenia is a severe mental health condition that requires a long and complicated diagnostic process. Past attempts to use deep learning for schizophrenia diagnosis from brain-imaging data have shown promise but suffer from a large training-application gap. We propose to reduce this training-application gap by focusing on readily accessible data.
  arXiv  Detail & Related papers  (2022-03-31T15:01:35Z)
• A Multimodal Approach for Automatic Mania Assessment in Bipolar Disorder [0.0]
  We create a multimodal decision system based on recordings of the patient in acoustic, linguistic, and visual modalities. We achieve a 64.8% unweighted average recall score, which improves the state-of-the-art performance achieved on this dataset.
  arXiv  Detail & Related papers  (2021-12-17T12:09:01Z)
• EEG functional connectivity and deep learning for automatic diagnosis of brain disorders: Alzheimer's disease and schizophrenia [0.0]
  We present a method for the automatic diagnosis of mental disorders based on the matrix of connections obtained from EEG time series and deep learning. We show that our approach can classify patients with Alzheimer's disease and schizophrenia with a high level of accuracy.
  arXiv  Detail & Related papers  (2021-10-07T23:26:38Z)
• Meta-learning on Spectral Images of Electroencephalogram of Schizophenics [0.0]
  Schizophrenia is a complex psychiatric disorder involving changes in thought patterns, perception, mood, and behavior. Advances in neuroimaging and machine learning algorithms can facilitate the diagnosis of schizophrenia.
  arXiv  Detail & Related papers  (2021-01-27T20:51:25Z)
• Deep Multi-task Learning for Depression Detection and Prediction in Longitudinal Data [50.02223091927777]
  Depression is among the most prevalent mental disorders, affecting millions of people of all ages globally. Machine learning techniques have shown effective in enabling automated detection and prediction of depression for early intervention and treatment. We introduce a novel deep multi-task recurrent neural network to tackle this challenge, in which depression classification is jointly optimized with two auxiliary tasks.
  arXiv  Detail & Related papers  (2020-12-05T05:14:14Z)
• Early Autism Spectrum Disorders Diagnosis Using Eye-Tracking Technology [62.997667081978825]
  Lack of money, absence of qualified specialists, and low level of trust to the correction methods are the main issues that affect the in-time diagnoses of ASD. Our team developed the algorithm that will be able to predict the chances of ASD according to the information from the gaze activity of the child.
  arXiv  Detail & Related papers  (2020-08-21T20:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.