Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making

• URL: http://arxiv.org/abs/2305.17588v3
• Date: Mon, 26 Feb 2024 23:11:03 GMT
• Title: Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
• Authors: Aliyah R. Hsu, Yeshwanth Cherapanamjeri, Briton Park, Tristan Naumann, Anobel Y. Odisho, Bin Yu
• Abstract summary: Pre-trained transformers are often fine-tuned to aid clinical decision-making using limited clinical notes. Model interpretability is crucial, especially in high-stakes domains like medicine, to establish trust and ensure safety. We introduce SUFO, a systematic framework that enhances interpretability of fine-tuned transformer feature spaces.
• Score: 14.377412942836143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pre-trained transformers are often fine-tuned to aid clinical decision-making using limited clinical notes. Model interpretability is crucial, especially in high-stakes domains like medicine, to establish trust and ensure safety, which requires human engagement. We introduce SUFO, a systematic framework that enhances interpretability of fine-tuned transformer feature spaces. SUFO utilizes a range of analytic and visualization techniques, including Supervised probing, Unsupervised similarity analysis, Feature dynamics, and Outlier analysis to address key questions about model trust and interpretability. We conduct a case study investigating the impact of pre-training data where we focus on real-world pathology classification tasks, and validate our findings on MedNLI. We evaluate five 110M-sized pre-trained transformer models, categorized into general-domain (BERT, TNLR), mixed-domain (BioBERT, Clinical BioBERT), and domain-specific (PubMedBERT) groups. Our SUFO analyses reveal that: (1) while PubMedBERT, the domain-specific model, contains valuable information for fine-tuning, it can overfit to minority classes when class imbalances exist. In contrast, mixed-domain models exhibit greater resistance to overfitting, suggesting potential improvements in domain-specific model robustness; (2) in-domain pre-training accelerates feature disambiguation during fine-tuning; and (3) feature spaces undergo significant sparsification during this process, enabling clinicians to identify common outlier modes among fine-tuned models as demonstrated in this paper. These findings showcase the utility of SUFO in enhancing trust and safety when using transformers in medicine, and we believe SUFO can aid practitioners in evaluating fine-tuned language models for other applications in medicine and in more critical domains.

Related papers

• Exploring Accurate and Transparent Domain Adaptation in Predictive Healthcare via Concept-Grounded Orthogonal Inference [7.191139788777488]
  ExtraCare decomposes patient representations into invariant and covariant components.<n>It offers human-understandable explanations by mapping sparse latent dimensions to medical concepts.<n>ExtraCare is evaluated on two real-world EHR datasets across multiple domain partition settings.
  arXiv  Detail & Related papers  (2026-02-13T02:46:50Z)
• MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement [63.82954136824963]
  Medical Vision-Language Models excel at perception tasks with complex clinical reasoning required in real-world scenarios.<n>We propose a novel reasoning MedVLM that addresses these challenges through domain-specific adaptation and guideline reinforcement.
  arXiv  Detail & Related papers  (2026-01-16T02:32:07Z)
• Foundation Models in Biomedical Imaging: Turning Hype into Reality [17.139610489482262]
  Foundation models (FMs) are driving a prominent shift in artificial intelligence across different domains, including biomedical imaging.<n>We critically assess the current state-of-the-art, analyzing hype by examining the core capabilities and limitations of FMs in the biomedical domain.<n>We discuss the paramount issues in deployment stemming from trustworthiness, bias, and safety, dissecting the challenges of algorithmic bias, data bias and privacy, and model hallucinations.
  arXiv  Detail & Related papers  (2025-12-17T05:18:43Z)
• Adversarial Hospital-Invariant Feature Learning for WSI Patch Classification [1.3955246652599635]
  We present the first systematic study of domain bias in pathology foundation models (PFMs) arising from hospital source characteristics.<n>We propose a lightweight adversarial framework that removes latent hospital-specific features from frozen representations without modifying the encoder itself.
  arXiv  Detail & Related papers  (2025-08-20T15:25:16Z)
• Decentralized LoRA Augmented Transformer with Context-aware Multi-scale Feature Learning for Secured Eye Diagnosis [2.1358421658740214]
  This paper proposes a novel Data efficient Image Transformer (DeiT) based framework that integrates context aware multiscale patch embedding, Low-Rank Adaptation (LoRA), knowledge distillation, and federated learning to address these challenges in a unified manner.<n>The proposed model effectively captures both local and global retinal features by leveraging multi scale patch representations with local and global attention mechanisms.
  arXiv  Detail & Related papers  (2025-05-11T13:51:56Z)
• Sparseformer: a Transferable Transformer with Multi-granularity Token Sparsification for Medical Time Series Classification [25.47662257105448]
  We introduce Sparseformer, a transformer specifically designed for MedTS classification. We propose a sparse token-based dual-attention mechanism that enables global modeling and token compression. Our model outperforms 12 baselines across seven medical datasets under supervised learning.
  arXiv  Detail & Related papers  (2025-03-19T13:22:42Z)
• ActiveSSF: An Active-Learning-Guided Self-Supervised Framework for Long-Tailed Megakaryocyte Classification [3.6535793744942318]
  We propose the ActiveSSF framework, which integrates active learning with self-supervised pretraining. Experimental results on clinical megakaryocyte datasets demonstrate that ActiveSSF achieves state-of-the-art performance. To foster further research, the code and datasets will be publicly released in the future.
  arXiv  Detail & Related papers  (2025-02-12T08:24:36Z)
• Transfer Learning with Clinical Concept Embeddings from Large Language Models [4.838020198075334]
  Large Language Models (LLMs) show significant potential of capturing the semantic meaning of clinical concepts. This study analyzed electronic health records from two large healthcare systems to assess the impact of semantic embeddings on local, shared, and transfer learning models.
  arXiv  Detail & Related papers  (2024-09-20T20:50:55Z)
• Analyzing the Effect of $k$-Space Features in MRI Classification Models [0.0]
  We have developed an explainable AI methodology tailored for medical imaging. We employ a Convolutional Neural Network (CNN) that analyzes MRI scans across both image and frequency domains. This approach not only enhances early training efficiency but also deepens our understanding of how additional features impact the model predictions.
  arXiv  Detail & Related papers  (2024-09-20T15:43:26Z)
• Improved EATFormer: A Vision Transformer for Medical Image Classification [0.0]
  This paper presents an improved Algorithm-based Transformer architecture for medical image classification using Vision Transformers. The proposed EATFormer architecture combines the strengths of Convolutional Neural Networks and Vision Transformers. Experimental results on the Chest X-ray and Kvasir datasets demonstrate that the proposed EATFormer significantly improves prediction speed and accuracy compared to baseline models.
  arXiv  Detail & Related papers  (2024-03-19T21:40:20Z)
• How Well Does GPT-4V(ision) Adapt to Distribution Shifts? A Preliminary Investigation [90.93999543169296]
  GPT-4V acts as the most advanced publicly accessible multimodal foundation model. This study rigorously evaluates GPT-4V's adaptability and generalization capabilities in dynamic environments.
  arXiv  Detail & Related papers  (2023-12-12T16:48:07Z)
• DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications [54.93807822347193]
  We show how to adapt attribution robustness estimation methods to a given domain, so as to take into account domain-specific plausibility. Next, we provide two methods, adversarial training and FAR training, to mitigate the brittleness characterized by DARE. Finally, we empirically validate our methods with extensive experiments on three established biomedical benchmarks.
  arXiv  Detail & Related papers  (2023-07-05T08:11:40Z)
• Curriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation [18.830738606514736]
  This work proposes the Curriculum-based Augmented Fourier Domain Adaptation (Curri-AFDA) for robust medical image segmentation. In particular, our curriculum learning strategy is based on the causal relationship of a model under different levels of data shift. Experiments on two segmentation tasks of Retina and Nuclei collected from multiple sites and scanners suggest that our proposed method yields superior adaptation and generalization performance.
  arXiv  Detail & Related papers  (2023-06-06T08:56:58Z)
• Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation [10.441315305453504]
  We propose a parameter-efficient transformer to explore intrinsic inductive bias via position information for medical image segmentation. Motivated by this, we present a novel Hybrid Axial-Attention (HAA) that can be equipped with spatial pixel-wise information and relative position information as inductive bias.
  arXiv  Detail & Related papers  (2022-11-17T13:54:55Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Transformer Based Multi-Source Domain Adaptation [53.24606510691877]
  In practical machine learning settings, the data on which a model must make predictions often come from a different distribution than the data it was trained on. Here, we investigate the problem of unsupervised multi-source domain adaptation, where a model is trained on labelled data from multiple source domains and must make predictions on a domain for which no labelled data has been seen. We show that the predictions of large pretrained transformer based domain experts are highly homogenous, making it challenging to learn effective functions for mixing their predictions.
  arXiv  Detail & Related papers  (2020-09-16T16:56:23Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.