Challenges in Explaining Pretrained Clinical Text Classifiers

• URL: http://arxiv.org/abs/2605.28060v1
• Date: Wed, 27 May 2026 07:09:16 GMT
• Title: Challenges in Explaining Pretrained Clinical Text Classifiers
• Abstract summary: We identify core limitations of token-level and perturbation-based explanation techniques through targeted demonstra- tions on a hospital length-of-stay prediction task.<n>Our findings reveal issues such as overemphasis on non-informative tokens, instability in at-tributions, and high-confidence predictions for incoherent input variants.
• Score: 0.3499870393443268
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explaining the predictions of neural models in clinical NLP remains a significant challenge, especially for complex tasks involving long, unstructured medical texts. While post-hoc methods like LIME and SHAP are widely used, they often fall short when applied to clinical narratives. In this paper, we identify core limitations of token-level and perturbation-based explanation techniques through targeted demonstra- tions on a hospital length-of-stay prediction task. Our findings reveal issues such as overemphasis on non-informative tokens, instability in at- tributions, and high-confidence predictions for incoherent input variants. These results underscore the need for explanation strategies that are clin- ically meaningful, semantically grounded, and robust to linguistic noise.

Related papers

• Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Models [67.51123543731806]
  Mental health has become a global priority, leading to a massive administrative burden in the coding of clinical diagnoses.<n>This study proposes the automation of psychiatric diagnostic analysis by mapping free-text descriptions to the International Classification of Diseases (ICD) using Natural Language Processing (NLP) and Machine Learning (ML) techniques.
  arXiv  Detail & Related papers  (2026-05-20T13:26:05Z)
• Scalable Token-Level Hallucination Detection in Large Language Models [63.3426544914783]
  Internal hallucinations are difficult to detect in reasoning-intensive tasks.<n>We propose TokenHD, a holistic pipeline for training token-level hallucination detectors.
  arXiv  Detail & Related papers  (2026-05-12T16:47:40Z)
• A BERT-Based Summarization approach for depression detection [1.7363112470483526]
  Depression is a globally prevalent mental disorder with potentially severe repercussions if not addressed. Machine learning and artificial intelligence can autonomously detect depression indicators from diverse data sources. Our study proposes text summarization as a preprocessing technique to diminish the length and intricacies of input texts.
  arXiv  Detail & Related papers  (2024-09-13T02:14:34Z)
• Uncertainty-aware Medical Diagnostic Phrase Identification and Grounding [72.18719355481052]
  We introduce a novel task called Medical Report Grounding (MRG)<n>MRG aims to directly identify diagnostic phrases and their corresponding grounding boxes from medical reports in an end-to-end manner.<n>We propose uMedGround, a robust and reliable framework that leverages a multimodal large language model to predict diagnostic phrases.
  arXiv  Detail & Related papers  (2024-04-10T07:41:35Z)
• Depression Detection on Social Media with Large Language Models [12.666554631713417]
  Social media platforms present a valuable data source for early depression diagnosis.<n>We propose DORIS, a framework that leverages Large Language Models (LLMs)<n>To integrate medical knowledge, DORIS utilizes LLMs to annotate user texts against established medical diagnostic criteria.
  arXiv  Detail & Related papers  (2024-03-16T01:01:16Z)
• Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
  Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning. They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
  arXiv  Detail & Related papers  (2023-05-30T22:05:11Z)
• NeuroExplainer: Fine-Grained Attention Decoding to Uncover Cortical Development Patterns of Preterm Infants [73.85768093666582]
  We propose an explainable geometric deep network dubbed NeuroExplainer. NeuroExplainer is used to uncover altered infant cortical development patterns associated with preterm birth.
  arXiv  Detail & Related papers  (2023-01-01T12:48:12Z)
• Toward Knowledge-Driven Speech-Based Models of Depression: Leveraging Spectrotemporal Variations in Speech Vowels [10.961439164833891]
  Psychomotor retardation associated with depression has been linked with tangible differences in vowel production. This paper investigates a knowledge-driven machine learning (ML) method that integrates spectrotemporal information of speech at the vowel-level to identify the depression.
  arXiv  Detail & Related papers  (2022-10-05T19:57:53Z)
• An Interpretable End-to-end Fine-tuning Approach for Long Clinical Text [72.62848911347466]
  Unstructured clinical text in EHRs contains crucial information for applications including decision support, trial matching, and retrospective research. Recent work has applied BERT-based models to clinical information extraction and text classification, given these models' state-of-the-art performance in other NLP domains. In this work, we propose a novel fine-tuning approach called SnipBERT. Instead of using entire notes, SnipBERT identifies crucial snippets and feeds them into a truncated BERT-based model in a hierarchical manner.
  arXiv  Detail & Related papers  (2020-11-12T17:14:32Z)
• Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings [16.136832979324467]
  We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset. We identify dangerous latent relationships that are captured by the contextual word embeddings. We evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks.
  arXiv  Detail & Related papers  (2020-03-11T23:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.