Detecting Dementia from Speech and Transcripts using Transformers
- URL: http://arxiv.org/abs/2110.14769v1
- Date: Wed, 27 Oct 2021 21:00:01 GMT
- Title: Detecting Dementia from Speech and Transcripts using Transformers
- Authors: Loukas Ilias, Dimitris Askounis, John Psarras
- Abstract summary: Alzheimer's disease (AD) constitutes a neurodegenerative disease with serious consequences to peoples' everyday lives, if it is not diagnosed early since there is no available cure.
Current work has been focused on diagnosing dementia from spontaneous speech.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Alzheimer's disease (AD) constitutes a neurodegenerative disease with serious
consequences to peoples' everyday lives, if it is not diagnosed early since
there is no available cure. Because of the cost of examinations for diagnosing
dementia, i.e., Magnetic Resonance Imaging (MRI), electroencephalogram (EEG)
signals etc., current work has been focused on diagnosing dementia from
spontaneous speech. However, little work has been done regarding the conversion
of speech data to Log-Mel spectrograms and Mel-frequency cepstral coefficients
(MFCCs) and the usage of pretrained models. Concurrently, little work has been
done in terms of both the usage of transformer networks and the way the two
modalities, i.e., speech and transcripts, are combined in a single neural
network. To address these limitations, first we employ several pretrained
models, with Vision Transformer (ViT) achieving the highest evaluation results.
Secondly, we propose multimodal models. More specifically, our introduced
models include Gated Multimodal Unit in order to control the influence of each
modality towards the final classification and crossmodal attention so as to
capture in an effective way the relationships between the two modalities.
Extensive experiments conducted on the ADReSS Challenge dataset demonstrate the
effectiveness of the proposed models and their superiority over
state-of-the-art approaches.
Related papers
- GFE-Mamba: Mamba-based AD Multi-modal Progression Assessment via Generative Feature Extraction from MCI [5.355943545567233]
Alzheimer's Disease (AD) is an irreversible neurodegenerative disorder that often progresses from Mild Cognitive Impairment (MCI)
We introduce GFE-Mamba, a classifier based on Generative Feature Extraction (GFE)
It integrates data from assessment scales, MRI, and PET, enabling deeper multimodal fusion.
Our experimental results demonstrate that the GFE-Mamba model is effective in predicting the conversion from MCI to AD.
arXiv Detail & Related papers (2024-07-22T15:22:33Z) - Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision.
This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
arXiv Detail & Related papers (2024-07-08T09:08:42Z) - Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection.
Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels.
Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
arXiv Detail & Related papers (2024-03-19T09:28:19Z) - Cross-modality Guidance-aided Multi-modal Learning with Dual Attention
for MRI Brain Tumor Grading [47.50733518140625]
Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly.
We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
arXiv Detail & Related papers (2024-01-17T07:54:49Z) - Diagnosing Alzheimer's Disease using Early-Late Multimodal Data Fusion
with Jacobian Maps [1.5501208213584152]
Alzheimer's disease (AD) is a prevalent and debilitating neurodegenerative disorder impacting a large aging population.
We propose an efficient early-late fusion (ELF) approach, which leverages a convolutional neural network for automated feature extraction and random forests.
To tackle the challenge of detecting subtle changes in brain volume, we transform images into the Jacobian domain (JD)
arXiv Detail & Related papers (2023-10-25T19:02:57Z) - Customizing General-Purpose Foundation Models for Medical Report
Generation [64.31265734687182]
The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks.
We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
arXiv Detail & Related papers (2023-06-09T03:02:36Z) - Context-aware attention layers coupled with optimal transport domain
adaptation and multimodal fusion methods for recognizing dementia from
spontaneous speech [0.0]
Alzheimer's disease (AD) constitutes a complex neurocognitive disease and is the main cause of dementia.
We propose some new methods for detecting AD patients, which capture the intra- and cross-modal interactions.
Experiments conducted on the ADReSS and ADReSSo Challenge indicate the efficacy of our introduced approaches over existing research initiatives.
arXiv Detail & Related papers (2023-05-25T18:18:09Z) - Improving Deep Facial Phenotyping for Ultra-rare Disorder Verification
Using Model Ensembles [52.77024349608834]
We analyze the influence of replacing a DCNN with a state-of-the-art face recognition approach, iResNet with ArcFace.
Our proposed ensemble model achieves state-of-the-art performance on both seen and unseen disorders.
arXiv Detail & Related papers (2022-11-12T23:28:54Z) - A Multimodal Approach for Dementia Detection from Spontaneous Speech
with Tensor Fusion Layer [0.0]
Alzheimer's disease (AD) is a progressive neurological disorder, which affects memory, thinking skills, and mental abilities.
We propose deep neural networks, which can be trained in an end-to-end trainable way and capture the inter- and intra-modal interactions.
arXiv Detail & Related papers (2022-11-08T16:43:58Z) - Differential Diagnosis of Frontotemporal Dementia and Alzheimer's
Disease using Generative Adversarial Network [0.0]
Frontotemporal dementia and Alzheimer's disease are two common forms of dementia and are easily misdiagnosed as each other.
Differentiating between the two dementia types is crucial for determining disease-specific intervention and treatment.
Recent development of Deep-learning-based approaches in the field of medical image computing are delivering some of the best performance for many binary classification tasks.
arXiv Detail & Related papers (2021-09-12T22:40:50Z) - Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for
Thoracic Disease Identification [83.6017225363714]
deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance.
For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming.
In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
arXiv Detail & Related papers (2021-02-26T02:29:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.