Multi-Label Classification with Generative AI Models in Healthcare: A Case Study of Suicidality and Risk Factors

• URL: http://arxiv.org/abs/2507.17009v1
• Date: Tue, 22 Jul 2025 20:44:44 GMT
• Title: Multi-Label Classification with Generative AI Models in Healthcare: A Case Study of Suicidality and Risk Factors
• Authors: Ming Huang, Zehan Li, Yan Hu, Wanjing Wang, Andrew Wen, Scott Lane, Salih Selek, Lokesh Shahani, Rodrigo Machado-Vieira, Jair Soares, Hua Xu, Hongfang Liu,
• Abstract summary: This study explores the use of generative large language models (LLMs), specifically GPT-3.5 and GPT-4.5, for multi-label classification (MLC) of suicidality-related factors (SrFs)<n>We present a novel end to end generative pipeline and introduce advanced evaluation methods, including label set level metrics and a multilabel confusion matrix for error analysis.<n>Our findings reveal systematic error patterns, such as the conflation of SI and SA, and highlight the models tendency toward cautious over labeling.
• Score: 9.89999399520808
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Suicide remains a pressing global health crisis, with over 720,000 deaths annually and millions more affected by suicide ideation (SI) and suicide attempts (SA). Early identification of suicidality-related factors (SrFs), including SI, SA, exposure to suicide (ES), and non-suicidal self-injury (NSSI), is critical for timely intervention. While prior studies have applied AI to detect SrFs in clinical notes, most treat suicidality as a binary classification task, overlooking the complexity of cooccurring risk factors. This study explores the use of generative large language models (LLMs), specifically GPT-3.5 and GPT-4.5, for multi-label classification (MLC) of SrFs from psychiatric electronic health records (EHRs). We present a novel end to end generative MLC pipeline and introduce advanced evaluation methods, including label set level metrics and a multilabel confusion matrix for error analysis. Finetuned GPT-3.5 achieved top performance with 0.94 partial match accuracy and 0.91 F1 score, while GPT-4.5 with guided prompting showed superior performance across label sets, including rare or minority label sets, indicating a more balanced and robust performance. Our findings reveal systematic error patterns, such as the conflation of SI and SA, and highlight the models tendency toward cautious over labeling. This work not only demonstrates the feasibility of using generative AI for complex clinical classification tasks but also provides a blueprint for structuring unstructured EHR data to support large scale clinical research and evidence based medicine.

Related papers

• LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLP [2.2615384250361004]
  This study introduces a novel LLM-augmented clinical NLP pipeline that employs domain-adapted large language models for symptom extraction, contextual reasoning, and correlation from free-text reports.<n> Evaluations on MIMIC-III and CARDIO-NLP datasets demonstrate improved performance in precision, recall, F1-score, and AUROC, with high clinical relevance.
  arXiv  Detail & Related papers  (2025-07-15T07:32:16Z)
• A Weakly Supervised Transformer to Support Rare Disease Diagnosis from Electronic Health Records: Methods and Applications in Rare Pulmonary Disease [16.112294460618955]
  Rare diseases affect an estimated 300-400 million people worldwide.<n> computational phenotyping algorithms show promise for rare disease detection.<n>We propose a weakly supervised, transformer-based framework that combines a small set of gold-standard labels with a large volume of iteratively updated silver-standard labels.
  arXiv  Detail & Related papers  (2025-07-01T23:11:20Z)
• Adaptable Cardiovascular Disease Risk Prediction from Heterogeneous Data using Large Language Models [70.64969663547703]
  AdaCVD is an adaptable CVD risk prediction framework built on large language models extensively fine-tuned on over half a million participants from the UK Biobank.<n>It addresses key clinical challenges across three dimensions: it flexibly incorporates comprehensive yet variable patient information; it seamlessly integrates both structured data and unstructured text; and it rapidly adapts to new patient populations using minimal additional data.
  arXiv  Detail & Related papers  (2025-05-30T14:42:02Z)
• Detecting PTSD in Clinical Interviews: A Comparative Analysis of NLP Methods and Large Language Models [6.916082619621498]
  Post-Traumatic Stress Disorder (PTSD) remains underdiagnosed in clinical settings.<n>This study evaluates natural language processing approaches for detecting PTSD from clinical interview transcripts.
  arXiv  Detail & Related papers  (2025-04-01T22:06:28Z)
• A Vector-Quantized Foundation Model for Patient Behavior Monitoring [43.02353546717171]
  This paper introduces a novel foundation model based on a modified vector quantized variational autoencoder, specifically designed to process real-world data from smartphones and wearable devices.<n>We leveraged the discrete latent representation of this model to effectively perform two downstream tasks, suicide risk assessment and emotional state prediction, on different held-out clinical cohorts without the need of fine-tuning.
  arXiv  Detail & Related papers  (2025-03-19T14:01:16Z)
• Suicide Phenotyping from Clinical Notes in Safety-Net Psychiatric Hospital Using Multi-Label Classification with Pre-Trained Language Models [10.384299115679369]
  Pre-trained language models offer promise for identifying suicidality from unstructured clinical narratives. We evaluated the performance of four BERT-based models using two fine-tuning strategies. The findings highlight that the model optimization, pretraining with domain-relevant data, and the single multi-label classification strategy enhance the model performance of suicide phenotyping.
  arXiv  Detail & Related papers  (2024-09-27T16:13:38Z)
• Optimizing Mortality Prediction for ICU Heart Failure Patients: Leveraging XGBoost and Advanced Machine Learning with the MIMIC-III Database [1.5186937600119894]
  Heart failure affects millions of people worldwide, significantly reducing quality of life and leading to high mortality rates. Despite extensive research, the relationship between heart failure and mortality rates among ICU patients is not fully understood. This study analyzed data from 1,177 patients over 18 years old from the MIMIC-III database, identified using ICD-9 codes.
  arXiv  Detail & Related papers  (2024-09-03T07:57:08Z)
• SOS-1K: A Fine-grained Suicide Risk Classification Dataset for Chinese Social Media Analysis [22.709733830774788]
  This study presents a Chinese social media dataset designed for fine-grained suicide risk classification. Seven pre-trained models were evaluated in two tasks: high and low suicide risk, and fine-grained suicide risk classification on a level of 0 to 10. Deep learning models show good performance in distinguishing between high and low suicide risk, with the best model achieving an F1 score of 88.39%.
  arXiv  Detail & Related papers  (2024-04-19T06:58:51Z)
• Fairness Evolution in Continual Learning for Medical Imaging [47.52603262576663]
  This study examines how bias evolves across tasks using domain-specific fairness metrics and how different CL strategies impact this evolution.<n>Our results show that Learning without Forgetting and Pseudo-Label achieve optimal classification performance, but Pseudo-Label is less biased.
  arXiv  Detail & Related papers  (2024-04-10T09:48:52Z)
• Multimodal Pretraining of Medical Time Series and Notes [45.89025874396911]
  Deep learning models show promise in extracting meaningful patterns, but they require extensive labeled data. We propose a novel approach employing self-supervised pretraining, focusing on the alignment of clinical measurements and notes. In downstream tasks, including in-hospital mortality prediction and phenotyping, our model outperforms baselines in settings where only a fraction of the data is labeled.
  arXiv  Detail & Related papers  (2023-12-11T21:53:40Z)
• Bag of Tricks for Long-Tailed Multi-Label Classification on Chest X-Rays [40.11576642444264]
  This report presents a brief description of our solution in the ICCV CVAMD 2023 CXR-LT Competition. We empirically explored the effectiveness for CXR diagnosis with the integration of several advanced designs. Our framework finally achieves 0.349 mAP on the competition test set, ranking in the top five.
  arXiv  Detail & Related papers  (2023-08-17T08:25:55Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• Am I No Good? Towards Detecting Perceived Burdensomeness and Thwarted Belongingness from Suicide Notes [51.378225388679425]
  We present an end-to-end multitask system to address a novel task of detection of Perceived Burdensomeness (PB) and Thwarted Belongingness (TB) from suicide notes. We also introduce a manually translated code-mixed suicide notes corpus, CoMCEASE-v2.0, based on the benchmark CEASE-v2.0 dataset. We exploit the temporal orientation and emotion information in the suicide notes to boost overall performance.
  arXiv  Detail & Related papers  (2022-05-20T06:31:08Z)
• SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models [48.374678491735665]
  We introduce Set-classifier with Class-specific RIsk Bounds (SCRIB) to tackle this problem. SCRIB constructs a set-classifier that controls the class-specific prediction risks with a theoretical guarantee. We validated SCRIB on several medical applications, including sleep staging on electroencephalogram (EEG) data, X-ray COVID image classification, and atrial fibrillation detection based on electrocardiogram (ECG) data.
  arXiv  Detail & Related papers  (2021-03-05T21:06:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.