ALBA: Adaptive Language-based Assessments for Mental Health

• URL: http://arxiv.org/abs/2311.06467v2
• Date: Thu, 16 May 2024 07:13:10 GMT
• Title: ALBA: Adaptive Language-based Assessments for Mental Health
• Authors: Vasudha Varadarajan, Sverker Sikström, Oscar N. E. Kjell, H. Andrew Schwartz,
• Abstract summary: This work introduces the task of Adaptive Language-Based Assessment ALBA. It involves adaptively ordering questions while also scoring an individual's latent psychological trait using limited language responses to previous questions. We found ALIRT to be the most accurate and scalable, achieving the highest accuracy with fewer questions.
• Score: 7.141164121152202
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Mental health issues differ widely among individuals, with varied signs and symptoms. Recently, language-based assessments have shown promise in capturing this diversity, but they require a substantial sample of words per person for accuracy. This work introduces the task of Adaptive Language-Based Assessment ALBA, which involves adaptively ordering questions while also scoring an individual's latent psychological trait using limited language responses to previous questions. To this end, we develop adaptive testing methods under two psychometric measurement theories: Classical Test Theory and Item Response Theory. We empirically evaluate ordering and scoring strategies, organizing into two new methods: a semi-supervised item response theory-based method ALIRT and a supervised Actor-Critic model. While we found both methods to improve over non-adaptive baselines, We found ALIRT to be the most accurate and scalable, achieving the highest accuracy with fewer questions (e.g., Pearson r ~ 0.93 after only 3 questions as compared to typically needing at least 7 questions). In general, adaptive language-based assessments of depression and anxiety were able to utilize a smaller sample of language without compromising validity or large computational costs.

Related papers

• Do LLMs Give Psychometrically Plausible Responses in Educational Assessments? [24.31027563947265]
  Knowing how test takers answer items in educational assessments is essential for test development.<n>If large language models (LLMs) exhibit human-like response behavior to test items, this could open up the possibility of using them as pilot participants to accelerate test development.
  arXiv  Detail & Related papers  (2025-06-11T14:41:10Z)
• TestAgent: An Adaptive and Intelligent Expert for Human Assessment [62.060118490577366]
  We propose TestAgent, a large language model (LLM)-powered agent designed to enhance adaptive testing through interactive engagement.<n>TestAgent supports personalized question selection, captures test-takers' responses and anomalies, and provides precise outcomes through dynamic, conversational interactions.
  arXiv  Detail & Related papers  (2025-06-03T16:07:54Z)
• Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practice [2.9775344067885974]
  We propose a novel adaptive Retrieval-Augmented Generation (RAG) approach that completes psychological questionnaires by analyzing social media posts. Our method retrieves the most relevant user posts for each question in a psychological survey and uses Large Language Models (LLMs) to predict questionnaire scores in a zero-shot setting.
  arXiv  Detail & Related papers  (2025-01-02T00:01:54Z)
• Detecting anxiety and depression in dialogues: a multi-label and explainable approach [5.635300481123079]
  Anxiety and depression are the most common mental health issues worldwide, affecting a non-negligible part of the population. In this work, an entirely novel system for the multi-label classification of anxiety and depression is proposed.
  arXiv  Detail & Related papers  (2024-12-23T15:29:46Z)
• Large Language Models as Neurolinguistic Subjects: Discrepancy in Performance and Competence for Form and Meaning [49.60849499134362]
  This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers. We found: (1) Psycholinguistic and neurolinguistic methods reveal that language performance and competence are distinct; (2) Direct probability measurement may not accurately assess linguistic competence; and (3) Instruction tuning won't change much competence but improve performance.
  arXiv  Detail & Related papers  (2024-11-12T04:16:44Z)
• LLM Questionnaire Completion for Automatic Psychiatric Assessment [49.1574468325115]
  We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C)
  arXiv  Detail & Related papers  (2024-06-09T09:03:11Z)
• Adapting Mental Health Prediction Tasks for Cross-lingual Learning via Meta-Training and In-context Learning with Large Language Model [3.3590922002216193]
  We use model-agnostic meta-learning and leveraging large language models (LLMs) to address this gap. We first apply a meta-learning model with self-supervision, which results in improved model initialisation for rapid adaptation and cross-lingual transfer. In parallel, we use LLMs' in-context learning capabilities to assess their performance accuracy across the Swahili mental health prediction tasks.
  arXiv  Detail & Related papers  (2024-04-13T17:11:35Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• SOUL: Towards Sentiment and Opinion Understanding of Language [96.74878032417054]
  We propose a new task called Sentiment and Opinion Understanding of Language (SOUL) SOUL aims to evaluate sentiment understanding through two subtasks: Review (RC) and Justification Generation (JG)
  arXiv  Detail & Related papers  (2023-10-27T06:48:48Z)
• Investigating Large Language Models' Perception of Emotion Using Appraisal Theory [3.0902630634005797]
  Large Language Models (LLM) have significantly advanced in recent years and are now being used by the general public. In this work, we investigate their emotion perception through the lens of appraisal and coping theory. We applied SCPQ to three recent LLMs from OpenAI, davinci-003, ChatGPT, and GPT-4 and compared the results with predictions from the appraisal theory and human data.
  arXiv  Detail & Related papers  (2023-10-03T16:34:47Z)
• Semantic Coherence Markers for the Early Diagnosis of the Alzheimer Disease [0.0]
  Perplexity was originally conceived as an information-theoretic measure to assess how much a given language model is suited to predict a text sequence. We employed language models as diverse as N-grams, from 2-grams to 5-grams, and GPT-2, a transformer-based language model. Best performing models achieved full accuracy and F-score (1.00 in both precision/specificity and recall/sensitivity) in categorizing subjects from both the AD class and control subjects.
  arXiv  Detail & Related papers  (2023-02-02T11:40:16Z)
• Adaptive Identification of Populations with Treatment Benefit in Clinical Trials: Machine Learning Challenges and Solutions [78.31410227443102]
  We study the problem of adaptively identifying patient subpopulations that benefit from a given treatment during a confirmatory clinical trial. We propose AdaGGI and AdaGCPI, two meta-algorithms for subpopulation construction.
  arXiv  Detail & Related papers  (2022-08-11T14:27:49Z)
• Predicting Human Psychometric Properties Using Computational Language Models [5.806723407090421]
  Transformer-based language models (LMs) continue to achieve state-of-the-art performance on natural language processing (NLP) benchmarks. Can LMs be of use in predicting the psychometric properties of test items, when those items are given to human participants? We gather responses from numerous human participants and LMs on a broad diagnostic test of linguistic competencies. We then use the human responses to calculate standard psychometric properties of the items in the diagnostic test, using the human responses and the LM responses separately.
  arXiv  Detail & Related papers  (2022-05-12T16:40:12Z)
• Delving Deeper into Cross-lingual Visual Question Answering [115.16614806717341]
  We show that simple modifications to the standard training setup can substantially reduce the transfer gap to monolingual English performance. We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers.
  arXiv  Detail & Related papers  (2022-02-15T18:22:18Z)
• AdaVQA: Overcoming Language Priors with Adapted Margin Cosine Loss [73.65872901950135]
  This work attempts to tackle the language prior problem from the viewpoint of the feature space learning. An adapted margin cosine loss is designed to discriminate the frequent and the sparse answer feature space. Experimental results demonstrate that our adapted margin cosine loss can greatly enhance the baseline models.
  arXiv  Detail & Related papers  (2021-05-05T11:41:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.