A Comprehensive Survey of Medical Image Segmentation: Challenges, Benchmarks, and Beyond

• URL: http://arxiv.org/abs/2606.16153v1
• Date: Mon, 15 Jun 2026 03:15:19 GMT
• Title: A Comprehensive Survey of Medical Image Segmentation: Challenges, Benchmarks, and Beyond
• Abstract summary: Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification.<n>This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U-Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives.<n>This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository.
• Score: 16.57577230673344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification. This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U-Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives. Unlike surveys that focus on a single model family or a specific clinical application, this review organizes U-Net-, Transformer-, and SAM-based methods within a unified analytical framework, with a particular focus on their effectiveness in improving segmentation accuracy and efficiency. This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository: https://github.com/andrew-pengyu/Awsome_MedSeg/tree/main.

Related papers

• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• AgentsEval: Clinically Faithful Evaluation of Medical Imaging Reports via Multi-Agent Reasoning [73.50200033931148]
  We introduce AgentsEval, a multi-agent stream reasoning framework that emulates the collaborative diagnostic workflow of radiologists.<n>By dividing the evaluation process into interpretable steps including criteria definition, evidence extraction, alignment, and consistency scoring, AgentsEval provides explicit reasoning traces and structured clinical feedback.<n> Experimental results demonstrate that AgentsEval delivers clinically aligned, semantically faithful, and interpretable evaluations that remain robust under paraphrastic, semantic, and stylistic perturbations.
  arXiv  Detail & Related papers  (2026-01-23T11:59:13Z)
• Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks [54.00822479127598]
  We introduce a medical vision-language task named Medical Diagnosis (MDS)<n>MDS aims to understand clinical queries for medical images and generate the corresponding segmentation masks as well as diagnostic results.<n>We propose Sim4Seg, a novel framework that improves the performance of diagnosis segmentation.
  arXiv  Detail & Related papers  (2025-11-10T03:22:42Z)
• Foundation Models in Medical Image Analysis: A Systematic Review and Meta-Analysis [7.905460364844281]
  Foundations models (FMs) have revolutionized medical image analysis, demonstrating strong zero- and few-shot performance across diverse medical imaging tasks.<n>FMs leverage large corpora of labeled and unlabeled multimodal datasets to learn generalized representations.<n>Despite the rapid proliferation of FM research in medical imaging, the field remains fragmented.<n>This review article provides a comprehensive and structured analysis of FMs in medical image analysis.
  arXiv  Detail & Related papers  (2025-10-19T19:19:23Z)
• Position: Thematic Analysis of Unstructured Clinical Transcripts with Large Language Models [5.398283020969301]
  Large language models (LLMs) can support thematic analysis of unstructured clinical transcripts.<n>Existing evaluation methods vary widely, hindering progress and preventing meaningful benchmarking across studies.<n>We propose an evaluation framework centered on three dimensions: validity, reliability, and interpretability.
  arXiv  Detail & Related papers  (2025-09-18T04:02:00Z)
• A Comprehensive Review of U-Net and Its Variants: Advances and Applications in Medical Image Segmentation [0.0]
  This paper classifies medical image datasets on the basis of their imaging modalities and examines U-Net and its various improvement models.<n>We summarize the four central improvement mechanisms of the U-Net and U-Net variant algorithms.<n>We propose potential avenues and strategies for future advancements.
  arXiv  Detail & Related papers  (2025-02-09T13:11:51Z)
• Named Clinical Entity Recognition Benchmark [2.9332007863461893]
  This report introduces a Named Clinical Entity Recognition Benchmark. It addresses the crucial natural language processing (NLP) task of extracting structured information from clinical narratives. The leaderboard provides a standardized platform for assessing diverse language models.
  arXiv  Detail & Related papers  (2024-10-07T14:00:18Z)
• Exploring SAM Ablations for Enhancing Medical Segmentation in Radiology and Pathology [2.5462695047893025]
  The Segment Anything Model (SAM) has emerged as a promising framework for addressing segmentation challenges across different domains. We explore the fine-tuning of SAM and assess its profound impact on the accuracy and reliability of segmentation results. We aim to bridge the gap between advanced segmentation techniques and the demanding requirements of healthcare.
  arXiv  Detail & Related papers  (2023-09-30T21:58:12Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)
• Weakly supervised multiple instance learning histopathological tumor segmentation [51.085268272912415]
  We propose a weakly supervised framework for whole slide imaging segmentation. We exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset.
  arXiv  Detail & Related papers  (2020-04-10T13:12:47Z)
• Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
  Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
  arXiv  Detail & Related papers  (2020-03-23T14:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.