Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification on the DAIC-WOZ

• URL: http://arxiv.org/abs/2407.19340v5
• Date: Fri, 11 Oct 2024 18:52:25 GMT
• Title: Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification on the DAIC-WOZ
• Abstract summary: Major Depressive Disorder (MDD) is a pervasive mental health condition that affects 300 million people worldwide. This work presents a novel, BiLSTM-based tri-modal model-level fusion architecture for the binary classification of depression from clinical interview recordings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Major Depressive Disorder (MDD) is a pervasive mental health condition that affects 300 million people worldwide. This work presents a novel, BiLSTM-based tri-modal model-level fusion architecture for the binary classification of depression from clinical interview recordings. The proposed architecture incorporates Mel Frequency Cepstral Coefficients, Facial Action Units, and uses a two-shot learning based GPT-4 model to process text data. This is the first work to incorporate large language models into a multi-modal architecture for this task. It achieves impressive results on the DAIC-WOZ AVEC 2016 Challenge cross-validation split and Leave-One-Subject-Out cross-validation split, surpassing all baseline models and multiple state-of-the-art models. In Leave-One-Subject-Out testing, it achieves an accuracy of 91.01%, an F1-Score of 85.95%, a precision of 80%, and a recall of 92.86%.

Related papers

• Towards Multidisciplinary Summarization of Hospital Stays: Efficient Sentence-Level Clinical Provenance Categorization [10.976481873409531]
  This study introduces a clinical provenance categorization pipeline using supervised fine-tuning (SFT) of large language models (LLMs)<n>We adapted two Llama-3 models to MedSecId, a corpus of 2,002 MIMIC-III (Adult ICU) notes annotated with clinical provenance headers.<n>We assessed model capacity (8B vs. 70B) and quantization on a gold-standard dataset of 227 sentence-level spans derived from three multi-disciplinary NICU summaries.
  arXiv  Detail & Related papers  (2026-06-01T16:57:51Z)
• CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification [0.5097809301149341]
  We introduce CMHL, a novel single-model architecture that explicitly models the logical structure of emotions.<n>With just 125M parameters, our model outperforms 56x larger LLMs and sLM ensembles.<n>Our work establishes that architectural intelligence (not parameter count) drives progress in Textual Emotion Classification.
  arXiv  Detail & Related papers  (2026-03-14T18:47:44Z)
• SYNAPSE-Net: A Unified Framework with Lesion-Aware Hierarchical Gating for Robust Segmentation of Heterogeneous Brain Lesions [0.23332469289621785]
  We propose the Unified Multi-Stream SYNAPSE-Net, an adaptive framework designed for both generalization and robustness.<n>The framework is built on a novel hybrid architecture integrating multi-stream CNN encoders, a Swin Transformer bottleneck for global context, and a dynamic cross-modal attention fusion mechanism.<n>The model was evaluated on three different challenging public datasets.
  arXiv  Detail & Related papers  (2025-10-30T19:40:42Z)
• A Novel Multi-branch ConvNeXt Architecture for Identifying Subtle Pathological Features in CT Scans [1.2461503242570642]
  This paper introduces a novel multi-branch ConvNeXt architecture designed specifically for the nuanced challenges of medical image analysis.<n>The proposed model incorporates a rigorous end-to-end pipeline, from meticulous data preprocessing to augmentation to a disciplined two-phase training strategy.<n> Experimental results demonstrate a superior performance on the validation set, achieving a final ROC-AUC of 0.9937, a validation accuracy of 0.9757, and an F1-score of 0.9825 for COVID-19 cases.
  arXiv  Detail & Related papers  (2025-10-10T08:00:46Z)
• Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction [95.91743732150233]
  Goedel-Prover-V2, a series of open-source language models, set a new state-of-the-art in automated theorem proving.<n>We generate synthetic tasks of increasing difficulty to train the model to master increasingly complex theorems.<n>Goedel-Prover-V2-32B achieves 88.1% on MiniF2F at pass@32 in standard mode and 90.4% in self-correction mode.
  arXiv  Detail & Related papers  (2025-08-05T16:28:22Z)
• Leveraging large language models and traditional machine learning ensembles for ADHD detection from narrative transcripts [6.55440666066668]
  We introduce an ensemble framework for automatically classifying Attention-Deficit/Hyperactivity Disorder (ADHD) diagnosis (binary) using narrative transcripts.<n>Our approach integrates three complementary models: LLaMA3, RoBERTa, and a Support Vector Machine (SVM)<n> Empirical results show that the ensemble outperforms individual models.
  arXiv  Detail & Related papers  (2025-05-27T15:22:01Z)
• Can Reasoning LLMs Enhance Clinical Document Classification? [7.026393789313748]
  Large Language Models (LLMs) offer promising improvements in accuracy and efficiency for this task. This study evaluates the performance and consistency of eight LLMs; four reasoning (Qwen QWQ, Deepseek Reasoner, GPT o3 Mini, Gemini 2.0 Flash Thinking) and four non-reasoning (Llama 3.3, GPT 4o Mini, Gemini 2.0 Flash, Deepseek Chat) Results showed that reasoning models outperformed non-reasoning models in accuracy (71% vs 68%) and F1 score (67% vs 60%)
  arXiv  Detail & Related papers  (2025-04-10T18:00:27Z)
• SemiHVision: Enhancing Medical Multimodal Models with a Semi-Human Annotated Dataset and Fine-Tuned Instruction Generation [13.672776832197918]
  Multimodal large language models (MLLMs) have made significant strides, yet they face challenges in the medical domain due to limited specialized knowledge. We seek to address this gap at various stages of the end-to-end learning pipeline, including data collection, model fine-tuning, and evaluation.
  arXiv  Detail & Related papers  (2024-10-19T02:35:35Z)
• Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation [41.25398139658467]
  Current foundation models have only been evaluated on a limited type and number of tasks, leaving their generalization ability and overall performance unclear. We establish a benchmark to evaluate the performance of off-the-shelf foundation models across six distinct clinical task types. We propose a unified knowledge distillation framework consisting of both expert and self-knowledge distillation, where the former allows the model to learn from the knowledge of multiple expert models.
  arXiv  Detail & Related papers  (2024-07-26T01:12:54Z)
• Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation [42.06416052431378]
  2D radiology captioning is incompetent to reflect the real-world diagnostic challenge in the volumetric 3D anatomy. We collected an 18,885 text-scan pairs 3D-BrainCT dataset and applied clinical visual instruction tuning to train BrainGPT models to generate radiology-adherent 3D brain CT reports. Our work embodies a holistic framework that showcased the first-hand experience of curating a 3D brain CT dataset, fine-tuning anatomy-sensible language models, and proposing robust radiology evaluation metrics.
  arXiv  Detail & Related papers  (2024-07-02T12:58:35Z)
• Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography [50.08496922659307]
  We propose a universal framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes. Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors.
  arXiv  Detail & Related papers  (2024-05-28T16:55:15Z)
• A Novel Fusion Architecture for PD Detection Using Semi-Supervised Speech Embeddings [8.996456485141069]
  We present a framework to recognize Parkinson's disease (PD) through an English pangram utterance speech collected using a web application. Our dataset includes a global cohort of 1306 participants, including 392 diagnosed with PD. We used deep learning embeddings derived from semi-supervised models such as Wav2Vec 2.0, WavLM, and ImageBind representing the speech dynamics associated with PD.
  arXiv  Detail & Related papers  (2024-05-21T16:06:51Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• Large Language Models to Identify Social Determinants of Health in Electronic Health Records [2.168737004368243]
  Social determinants of health (SDoH) have an important impact on patient outcomes but are incompletely collected from the electronic health records (EHRs) This study researched the ability of large language models to extract SDoH from free text in EHRs, where they are most commonly documented. 800 patient notes were annotated for SDoH categories, and several transformer-based models were evaluated.
  arXiv  Detail & Related papers  (2023-08-11T19:18:35Z)
• Predicting Issue Types with seBERT [85.74803351913695]
  seBERT is a model that was developed based on the BERT architecture, but trained from scratch with software engineering data. We fine-tuned this model for the NLBSE challenge for the task of issue type prediction. Our model dominates the baseline fastText for all three issue types in both recall and precisio to achieve an overall F1-score of 85.7%.
  arXiv  Detail & Related papers  (2022-05-03T06:47:13Z)
• RoBLEURT Submission for the WMT2021 Metrics Task [72.26898579202076]
  We present our submission to the Shared Metrics Task: RoBLEURT. Our model reaches state-of-the-art correlations with the WMT 2020 human annotations upon 8 out of 10 to-English language pairs.
  arXiv  Detail & Related papers  (2022-04-28T08:49:40Z)
• AI-enabled Automatic Multimodal Fusion of Cone-Beam CT and Intraoral Scans for Intelligent 3D Tooth-Bone Reconstruction and Clinical Applications [29.065668174732014]
  A critical step in virtual dental treatment planning is to accurately delineate all tooth-bone structures from CBCT. Previous studies have established several methods for CBCT segmentation using deep learning. Here, we present a Deep Dental Multimodal Analysis framework consisting of a CBCT segmentation model, an intraoral scan (IOS) segmentation model, and a fusion model to generate 3D fused crown-root-bone structures.
  arXiv  Detail & Related papers  (2022-03-11T07:50:15Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.