Benchmarking Foundation and Large Language Models for Few-Shot Medical Image Segmentation

• URL: http://arxiv.org/abs/2607.27856v1
• Date: Thu, 30 Jul 2026 08:34:36 GMT
• Title: Benchmarking Foundation and Large Language Models for Few-Shot Medical Image Segmentation
• Abstract summary: Few-shot medical image segmentation (FS-MIS) aims to segment novel regions of interest from a few annotated support examples.<n>Existing FS-MIS solutions span diverse paradigms but are evaluated under inconsistent settings.<n>We introduce FAME, a unified benchmark for evaluating FS-MIS solutions.
• Score: 6.485015749813599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot medical image segmentation (FS-MIS) aims to segment novel regions of interest (ROIs) from a few annotated support examples. Despite rapid progress, existing FS-MIS solutions span diverse paradigms but are evaluated under inconsistent settings, leaving their relative effectiveness unclear. We introduce FAME, a unified benchmark for evaluating FS-MIS solutions, covering specialists, SAM-based methods, CLIP-based methods, and MLLM-based methods. FAME contains 14,958 test samples across 7 anatomical sites, 9 imaging modalities, and 14 ROI categories, and evaluates models under zero-shot and ten-shot settings with additional assessment of target-absence recognition and generalization under covariate and semantic shifts. Our evaluation reveals several findings. First, effective few-shot segmentation depends on how models exploit support examples: direct visual adaptation generally outperforms prompt-based strategies. Second, increasing support examples improves performance only when models can effectively utilize them. Third, semantic transfer remains substantially more challenging than imaging-domain adaptation, and strong localization ability does not necessarily imply reliable target-absence recognition. We hope FAME provides a comprehensive understanding of current FS-MIS solutions and facilitates the development of more effective and reliable few-shot medical segmentation methods.

Related papers

• SD-FSMIS: Adapting Stable Diffusion for Few-Shot Medical Image Segmentation [4.580922649026417]
  Few-Shot Medical Image (FSMIS) aims to segment novel object classes in medical images using only minimal annotated examples.<n>We introduce SD-FSMIS, a novel framework designed to effectively adapt the powerful Stable Diffusion model for the FSMIS task.
  arXiv  Detail & Related papers  (2026-04-03T15:54:34Z)
• FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation [57.930067699003764]
  Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data.<n>We introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation.<n>We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-12T08:28:41Z)
• Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Models [15.976622378615714]
  Medical image segmentation plays a vital role in clinical decision-making, enabling precise localization of lesions and guiding interventions.<n>Current explanation techniques have primarily focused on classification tasks, leaving the segmentation domain relatively underexplored.<n>We introduce an explanation model for segmentation task which employs the causal inference framework and backpropagates the average treatment effect (ATE) into a metric to determine the influence of input regions, as well as network components, on target segmentation areas.
  arXiv  Detail & Related papers  (2026-02-24T03:26:27Z)
• Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design [8.993770750003673]
  Few-Shot Semantic (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples.<n>This paper introduces the first dedicated method for interpreting matching-based FSS models.<n>Our Affinity Explainer approach extracts attribution maps that highlight which pixels in support images contribute most to query segmentation predictions.
  arXiv  Detail & Related papers  (2025-11-22T19:22:10Z)
• Unified modality separation: A vision-language framework for unsupervised domain adaptation [60.8391821117794]
  Unsupervised domain adaptation (UDA) enables models trained on a labeled source domain to handle new unlabeled domains.<n>We propose a unified modality separation framework that accommodates both modality-specific and modality-invariant components.<n>Our methods achieve up to 9% performance gain with 9 times of computational efficiencies.
  arXiv  Detail & Related papers  (2025-08-07T02:51:10Z)
• Cross-Modal Clustering-Guided Negative Sampling for Self-Supervised Joint Learning from Medical Images and Reports [11.734906190235066]
  This paper presents a Cross-Modal Cluster-Guided Negative Sampling (CM-CGNS) method with two-fold ideas.<n>First, it extends the k-means clustering used for local text features in the single-modal domain to the multimodal domain through cross-modal attention.<n>Second, it introduces a Cross-Modal Masked Image Reconstruction (CM-MIR) module that leverages local text-to-image features obtained via cross-modal attention to reconstruct masked local image regions.
  arXiv  Detail & Related papers  (2025-06-13T11:08:16Z)
• One-shot In-context Part Segmentation [97.77292483684877]
  We present the One-shot In-context Part (OIParts) framework to tackle the challenges of part segmentation.<n>Our framework offers a novel approach to part segmentation that is training-free, flexible, and data-efficient.<n>We have achieved remarkable segmentation performance across diverse object categories.
  arXiv  Detail & Related papers  (2025-03-03T03:50:54Z)
• A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models [7.428199805959228]
  Few-shot semantic segmentation (FSS) is a crucial challenge in computer vision.<n>With the emergence of vision foundation models (VFM) as generalist feature extractors, we seek to explore the adaptation of these models for FSS.<n>We propose a novel realistic benchmark with a simple and straightforward adaptation process tailored for this task.
  arXiv  Detail & Related papers  (2024-01-20T19:50:51Z)
• On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation [47.95611203419802]
  Foundations for vision and language, pre-trained on extensive sets of natural image and text data, have emerged as a promising approach. We compare the generalization performance to unseen domains of various pre-trained models after being fine-tuned on the same in-distribution dataset. We further developed a new Bayesian uncertainty estimation for frozen models and used them as an indicator to characterize the model's performance on out-of-distribution data.
  arXiv  Detail & Related papers  (2023-11-18T14:52:10Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.