Feature Quality and Adaptability of Medical Foundation Models: A Comparative Evaluation for Radiographic Classification and Segmentation

• URL: http://arxiv.org/abs/2511.09742v1
• Date: Fri, 14 Nov 2025 01:07:27 GMT
• Title: Feature Quality and Adaptability of Medical Foundation Models: A Comparative Evaluation for Radiographic Classification and Segmentation
• Abstract summary: We evaluate vision encoders from eight medical and general-domain FMs for chest X-ray analysis.<n>We benchmark classification (pneumothorax, cardiomegaly) and segmentation (pneumothorax, cardiac boundary) using linear probing and fine-tuning.
• Score: 5.33356570264555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models (FMs) promise to generalize medical imaging, but their effectiveness varies. It remains unclear how pre-training domain (medical vs. general), paradigm (e.g., text-guided), and architecture influence embedding quality, hindering the selection of optimal encoders for specific radiology tasks. To address this, we evaluate vision encoders from eight medical and general-domain FMs for chest X-ray analysis. We benchmark classification (pneumothorax, cardiomegaly) and segmentation (pneumothorax, cardiac boundary) using linear probing and fine-tuning. Our results show that domain-specific pre-training provides a significant advantage; medical FMs consistently outperformed general-domain models in linear probing, establishing superior initial feature quality. However, feature utility is highly task-dependent. Pre-trained embeddings were strong for global classification and segmenting salient anatomy (e.g., heart). In contrast, for segmenting complex, subtle pathologies (e.g., pneumothorax), all FMs performed poorly without significant fine-tuning, revealing a critical gap in localizing subtle disease. Subgroup analysis showed FMs use confounding shortcuts (e.g., chest tubes for pneumothorax) for classification, a strategy that fails for precise segmentation. We also found that expensive text-image alignment is not a prerequisite; image-only (RAD-DINO) and label-supervised (Ark+) FMs were among top performers. Notably, a supervised, end-to-end baseline remained highly competitive, matching or exceeding the best FMs on segmentation tasks. These findings show that while medical pre-training is beneficial, architectural choices (e.g., multi-scale) are critical, and pre-trained features are not universally effective, especially for complex localization tasks where supervised models remain a strong alternative.

Related papers

• Foundation models for movement data: Are they ready for prime-time? [44.17303085911197]
  Foundation models (FMs) trained on large-scale accelerometer data have been proposed as general-purpose feature extractors for health monitoring.<n>We present the first comprehensive evaluation of four open-source accelerometer FMs against supervised baselines.
  arXiv  Detail & Related papers  (2026-08-13T14:39:53Z)
• SARFA: Segment Anything with Radiomic Feature Alignment [0.790660895390689]
  The Segment Anything Model (SAM) has demonstrated strong generalizability across a variety of segmentation tasks.<n>This poses a problem in medical imaging, where accurate delineation of targets such as tumors is vital.<n>We propose SARFA, a novel framework for improved medical image segmentation.
  arXiv  Detail & Related papers  (2026-07-14T23:09:10Z)
• A Robust Unsupervised Domain Adaptation Framework for Medical Image Classification Using RKHS-MMD [0.9634859579172255]
  Labeling medical images is a major bottleneck in the field of medical imaging.<n>We propose an unsupervised domain adaptation framework that combines transfer learning with a Reproducing Kernel Space based Maximum Mean Discrepancy loss.<n>We show that RKHS-MMD performs better than the standard Maximum Mean Discrepancy in reducing modality gap.
  arXiv  Detail & Related papers  (2026-05-05T14:14:25Z)
• SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation [7.174788000625294]
  Unsupervised Domain Adaptation (UDA) is essential for deploying medical segmentation models across diverse clinical environments.<n>Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation (SHAPE) is a framework that reframes adaptation towards global anatomical plausibility.
  arXiv  Detail & Related papers  (2026-03-23T12:25:56Z)
• CardioBench: Do Echocardiography Foundation Models Generalize Beyond the Lab? [4.445909648625997]
  Foundation models (FMs) are reshaping medical imaging, yet their application in echocardiography remains limited.<n>We introduce CardioBench, a comprehensive benchmark for echocardiography FMs.<n> CardioBench unifies eight publicly available datasets into a standardized suite spanning four regression and five classification tasks.
  arXiv  Detail & Related papers  (2025-10-01T05:09:48Z)
• Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis [4.803310914375717]
  This study evaluates three vision-language foundation models (RAD-DINO, CheXagent, and BiomedCLIP) on their ability to capture fine-grained imaging features for radiology tasks.<n>The models were assessed across classification, segmentation, and regression tasks for pneumothorax and cardiomegaly on chest radiographs.
  arXiv  Detail & Related papers  (2025-04-22T17:20:34Z)
• Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation? [10.20366295974822]
  We introduce a novel decode head architecture, HQHSAM, which simply integrates elements from two state-of-the-art decoder heads, HSAM and HQSAM, to enhance segmentation performance. Our experiments on multiple datasets, encompassing various anatomies and modalities, reveal that FMs, particularly with the HQHSAM decode head, improve domain generalization for medical image segmentation.
  arXiv  Detail & Related papers  (2024-09-12T11:41:35Z)
• Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning [65.54680361074882]
  Eye-gaze Guided Multi-modal Alignment (EGMA) framework harnesses eye-gaze data for better alignment of medical visual and textual features. We conduct downstream tasks of image classification and image-text retrieval on four medical datasets.
  arXiv  Detail & Related papers  (2024-03-19T03:59:14Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Model [4.397622801930704]
  Chest X-rays (CXRs) are a widely used imaging modality for the diagnosis and prognosis of lung disease. We propose a new self-supervised paradigm, where a general representation from CXRs is learned using a group-masked self-supervised framework. The pre-trained model is then fine-tuned for domain-specific tasks such as covid-19, pneumonia detection, and general health screening.
  arXiv  Detail & Related papers  (2022-11-23T13:38:16Z)
• Data-Efficient Vision Transformers for Multi-Label Disease Classification on Chest Radiographs [55.78588835407174]
  Vision Transformers (ViTs) have not been applied to this task despite their high classification performance on generic images. ViTs do not rely on convolutions but on patch-based self-attention and in contrast to CNNs, no prior knowledge of local connectivity is present. Our results show that while the performance between ViTs and CNNs is on par with a small benefit for ViTs, DeiTs outperform the former if a reasonably large data set is available for training.
  arXiv  Detail & Related papers  (2022-08-17T09:07:45Z)
• Preservation of High Frequency Content for Deep Learning-Based Medical Image Classification [74.84221280249876]
  An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information.
  arXiv  Detail & Related papers  (2022-05-08T15:29:54Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)
• Towards to Robust and Generalized Medical Image Segmentation Framework [17.24628770042803]
  We propose a novel two-stage framework for robust generalized segmentation. In particular, an unsupervised Tile-wise AutoEncoder (T-AE) pretraining architecture is coined to learn meaningful representation. Experiments of lung segmentation on multi chest X-ray datasets are conducted.
  arXiv  Detail & Related papers  (2021-08-09T05:58:49Z)
• Cross-Modal Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop [63.81818077092879]
  We propose an end-to-end semi-supervised cross-modal contrastive learning framework for medical images. We first apply an image encoder to classify the chest X-rays and to generate the image features. The radiomic features are then passed through another dedicated encoder to act as the positive sample for the image features generated from the same chest X-ray.
  arXiv  Detail & Related papers  (2021-04-11T09:16:29Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.