The Capability of Large Language Models to Measure Psychiatric
Functioning
- URL: http://arxiv.org/abs/2308.01834v1
- Date: Thu, 3 Aug 2023 15:52:27 GMT
- Title: The Capability of Large Language Models to Measure Psychiatric
Functioning
- Authors: Isaac R. Galatzer-Levy, Daniel McDuff, Vivek Natarajan, Alan
Karthikesalingam and Matteo Malgaroli
- Abstract summary: The Med-PaLM 2 is capable of assessing psychiatric functioning across a range of psychiatric conditions.
The strongest performance was the prediction of depression scores based on standardized assessments.
Results show the potential for general clinical language models to flexibly predict psychiatric risk.
- Score: 9.938814639951842
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The current work investigates the capability of Large language models (LLMs)
that are explicitly trained on large corpuses of medical knowledge (Med-PaLM 2)
to predict psychiatric functioning from patient interviews and clinical
descriptions without being trained to do so. To assess this, n = 145 depression
and n =115 PTSD assessments and n = 46 clinical case studies across high
prevalence/high comorbidity disorders (Depressive, Anxiety, Psychotic, trauma
and stress, Addictive disorders) were analyzed using prompts to extract
estimated clinical scores and diagnoses. Results demonstrate that Med-PaLM 2 is
capable of assessing psychiatric functioning across a range of psychiatric
conditions with the strongest performance being the prediction of depression
scores based on standardized assessments (Accuracy range= 0.80 - 0.84) which
were statistically indistinguishable from human clinical raters t(1,144) =
1.20; p = 0.23. Results show the potential for general clinical language models
to flexibly predict psychiatric risk based on free descriptions of functioning
from both patients and clinicians.
Related papers
- LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.
We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.
Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
arXiv Detail & Related papers (2025-01-07T08:49:04Z) - PSYCHE: A Multi-faceted Patient Simulation Framework for Evaluation of Psychiatric Assessment Conversational Agents [2.8216674865505627]
Psychiatric assessment conversational agents (PACAs) aim to simulate the role of psychiatrists in clinical evaluations.
Here, we propose PSYCHE, a novel framework designed to enable the 1) clinically relevant, 2) ethically safe, 3) cost-efficient, and 4) quantitative evaluation of PACAs.
This is achieved by simulating psychiatric patients based on a multi-faceted psychiatric construct that defines the simulated patients' profiles, histories, and behaviors.
arXiv Detail & Related papers (2025-01-03T01:38:46Z) - GPT-4 on Clinic Depression Assessment: An LLM-Based Pilot Study [0.6999740786886538]
We explore the use of GPT-4 for clinical depression assessment based on transcript analysis.
We examine the model's ability to classify patient interviews into binary categories: depressed and not depressed.
Results indicate that GPT-4 exhibits considerable variability in accuracy and F1-Score across configurations.
arXiv Detail & Related papers (2024-12-31T00:32:43Z) - Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities [25.305909441170993]
Depression has proven to be a significant public health issue, profoundly affecting the psychological well-being of individuals.
If it remains undiagnosed, depression can lead to severe health issues, which can manifest physically and even lead to suicide.
arXiv Detail & Related papers (2024-07-08T17:00:51Z) - SemioLLM: Assessing Large Language Models for Semiological Analysis in Epilepsy Research [45.2233252981348]
Large Language Models have shown promising results in their ability to encode general medical knowledge.
We test the ability of state-of-the-art LLMs to leverage their internal knowledge and reasoning for epilepsy diagnosis.
arXiv Detail & Related papers (2024-07-03T11:02:12Z) - LLM Questionnaire Completion for Automatic Psychiatric Assessment [49.1574468325115]
We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains.
The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C)
arXiv Detail & Related papers (2024-06-09T09:03:11Z) - Assessing ML Classification Algorithms and NLP Techniques for Depression Detection: An Experimental Case Study [0.6524460254566905]
Depression has affected millions of people worldwide and has become one of the most common mental disorders.
Recent research has evidenced that machine learning (ML) and Natural Language Processing (NLP) tools and techniques have significantly been used to diagnose depression.
However, there are still several challenges in the assessment of depression detection approaches in which other conditions such as post-traumatic stress disorder (PTSD) are present.
arXiv Detail & Related papers (2024-04-03T19:45:40Z) - Precision psychiatry: predicting predictability [0.0]
I review ten challenges in the field of precision psychiatry.
Need for studies on real-world populations and realistic clinical outcome definitions.
Consider treatment-related factors such as placebo effects and non-adherence to prescriptions.
arXiv Detail & Related papers (2023-06-21T13:10:46Z) - Handwriting and Drawing for Depression Detection: A Preliminary Study [53.11777541341063]
Short-term covid effects on mental health were a significant increase in anxiety and depressive symptoms.
The aim of this study is to use a new tool, the online handwriting and drawing analysis, to discriminate between healthy individuals and depressed patients.
arXiv Detail & Related papers (2023-02-05T22:33:49Z) - Clinical Outcome Prediction from Admission Notes using Self-Supervised
Knowledge Integration [55.88616573143478]
Outcome prediction from clinical text can prevent doctors from overlooking possible risks.
Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets.
We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
arXiv Detail & Related papers (2021-02-08T10:26:44Z) - Deep Multi-task Learning for Depression Detection and Prediction in
Longitudinal Data [50.02223091927777]
Depression is among the most prevalent mental disorders, affecting millions of people of all ages globally.
Machine learning techniques have shown effective in enabling automated detection and prediction of depression for early intervention and treatment.
We introduce a novel deep multi-task recurrent neural network to tackle this challenge, in which depression classification is jointly optimized with two auxiliary tasks.
arXiv Detail & Related papers (2020-12-05T05:14:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.