Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models

• URL: http://arxiv.org/abs/2604.07717v2
• Date: Sun, 12 Apr 2026 19:55:21 GMT
• Title: Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models
• Abstract summary: This study aims to develop a large language model (LLM)-based tool for identifying HIV stigma from clinical notes.<n>We identified clinical notes from PLWH receiving care at the University of Florida (UF) Health between 2012 and 2022.<n>A total of 1,332 sentences were manually annotated across four stigma subscales: Concern with Public Attitudes, Disclosure Concerns, Negative Self-Image, and Personalized Stigma.
• Score: 10.723891827798028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human immunodeficiency virus (HIV)-related stigma is a critical psychosocial determinant of health for people living with HIV (PLWH), influencing mental health, engagement in care, and treatment outcomes. Although stigma-related experiences are documented in clinical narratives, there is a lack of off-the-shelf tools to extract and categorize them. This study aims to develop a large language model (LLM)-based tool for identifying HIV stigma from clinical notes. We identified clinical notes from PLWH receiving care at the University of Florida (UF) Health between 2012 and 2022. Candidate sentences were identified using expert-curated stigma-related keywords and iteratively expanded via clinical word embeddings. A total of 1,332 sentences were manually annotated across four stigma subscales: Concern with Public Attitudes, Disclosure Concerns, Negative Self-Image, and Personalized Stigma. We compared GatorTron-large and BERT as encoder-based baselines, and GPT-OSS-20B, LLaMA-8B, and MedGemma-27B as generative LLMs, under zero-shot and few-shot prompting. GatorTron-large achieved the best overall performance (Micro F1 = 0.62). Few-shot prompting substantially improved generative model performance, with 5-shot GPT-OSS-20B and LLaMA-8B achieving Micro-F1 scores of 0.57 and 0.59, respectively. Performance varied by stigma subscale, with Negative Self-Image showing the highest predictability and Personalized Stigma remaining the most challenging. Zero-shot generative inference exhibited non-trivial failure rates (up to 32%). This study develops the first practical NLP tool for identifying HIV stigma in clinical notes.

Related papers

• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Models [67.51123543731806]
  Mental health has become a global priority, leading to a massive administrative burden in the coding of clinical diagnoses.<n>This study proposes the automation of psychiatric diagnostic analysis by mapping free-text descriptions to the International Classification of Diseases (ICD) using Natural Language Processing (NLP) and Machine Learning (ML) techniques.
  arXiv  Detail & Related papers  (2026-05-20T13:26:05Z)
• MedArena: Comparing LLMs for Medicine-in-the-Wild Clinician Preferences [50.71326426975699]
  We present MedArena, an interactive evaluation platform for medical large language models (LLMs)<n>Given a clinician-provided query, MedArena presents responses from two randomly selected models and asks the user to select the preferred response.<n>Out of 1571 preferences collected across 12 LLMs up to November 1, 2025, Gemini 2.0 Flash Thinking, Gemini 2.5 Pro, and GPT-4o were the top three models by Bradley-Terry rating.
  arXiv  Detail & Related papers  (2026-03-13T22:30:26Z)
• Fine-Tune, Don't Prompt, Your Language Model to Identify Biased Language in Clinical Notes [31.925112343488976]
  Clinical documentation can contain emotionally charged language with stigmatizing or privileging valences.<n>We present a framework for detecting and classifying such language.<n>Fine-tuning with lexically primed inputs consistently outperformed prompting approaches.
  arXiv  Detail & Related papers  (2026-02-16T22:39:28Z)
• BioPulse-QA: A Dynamic Biomedical Question-Answering Benchmark for Evaluating Factuality, Robustness, and Bias in Large Language Models [7.8780007697387235]
  We introduce BioPulse-QA, a benchmark that evaluates large language models (LLMs) on answering questions from newly published biomedical documents.<n>We evaluate four LLMs - GPT-o1, GPT-o1, Gemini-2.0-Flash, and LLaMA-3.1 8B Instruct - released prior to the publication dates of the benchmark documents.
  arXiv  Detail & Related papers  (2026-01-19T00:38:33Z)
• Multi-Label Classification with Generative AI Models in Healthcare: A Case Study of Suicidality and Risk Factors [9.89999399520808]
  This study explores the use of generative large language models (LLMs), specifically GPT-3.5 and GPT-4.5, for multi-label classification (MLC) of suicidality-related factors (SrFs)<n>We present a novel end to end generative pipeline and introduce advanced evaluation methods, including label set level metrics and a multilabel confusion matrix for error analysis.<n>Our findings reveal systematic error patterns, such as the conflation of SI and SA, and highlight the models tendency toward cautious over labeling.
  arXiv  Detail & Related papers  (2025-07-22T20:44:44Z)
• Application of CARE-SD text classifier tools to assess distribution of stigmatizing and doubt-marking language features in EHR [1.58767962323279]
  Stigmatizing language occurred at higher rates among historically stigmatized patients, perpetuated by multiple provider types.<n>We identified linguistic features of doubt markers and stigmatizing labels in MIMIC-III EHR.
  arXiv  Detail & Related papers  (2025-07-11T18:55:28Z)
• Enhanced Large Language Models for Effective Screening of Depression and Anxiety [44.81045754697482]
  This paper introduces a pipeline for synthesizing clinical interviews, resulting in 1,157 interactive dialogues (PsyInterview)<n>EmoScan distinguishes between coarse (e.g., anxiety or depressive disorders) and fine disorders (e.g., major depressive disorders) and conducts high-quality interviews.
  arXiv  Detail & Related papers  (2025-01-15T12:42:09Z)
• Suicide Phenotyping from Clinical Notes in Safety-Net Psychiatric Hospital Using Multi-Label Classification with Pre-Trained Language Models [10.384299115679369]
  Pre-trained language models offer promise for identifying suicidality from unstructured clinical narratives. We evaluated the performance of four BERT-based models using two fine-tuning strategies. The findings highlight that the model optimization, pretraining with domain-relevant data, and the single multi-label classification strategy enhance the model performance of suicide phenotyping.
  arXiv  Detail & Related papers  (2024-09-27T16:13:38Z)
• Words Matter: Reducing Stigma in Online Conversations about Substance Use with Large Language Models [0.0]
  Stigma is a barrier to treatment for individuals struggling with substance use disorders (SUD) This study investigates how stigma manifests on social media, particularly Reddit, where anonymity can exacerbate discriminatory behaviors.
  arXiv  Detail & Related papers  (2024-08-15T01:00:28Z)
• WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions [46.60244609728416]
  Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a litmus test of a model's utility in clinical practice. We introduce an evaluation design that focuses on the robustness and explainability of LMs in identifying Wellness Dimensions (WDs) We reveal four surprising results about LMs/LLMs.
  arXiv  Detail & Related papers  (2024-06-17T19:50:40Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.