Segment Anything in Pathology Images with Natural Language

• URL: http://arxiv.org/abs/2506.20988v2
• Date: Tue, 19 Aug 2025 03:06:01 GMT
• Title: Segment Anything in Pathology Images with Natural Language
• Authors: Zhixuan Chen, Junlin Hou, Liqi Lin, Yihui Wang, Yequan Bie, Xi Wang, Yanning Zhou, Ronald Cheong Kin Chan, Hao Chen,
• Abstract summary: PathSegmentor is the first text-prompted segmentation foundation model designed specifically for pathology images.<n>We also introduce PathSeg, the largest and most comprehensive dataset for pathology segmentation.
• Score: 10.525414795571393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pathology image segmentation is crucial in computational pathology for analyzing histological features relevant to cancer diagnosis and prognosis. However, current methods face major challenges in clinical applications due to limited annotated data and restricted category definitions. To address these limitations, we propose PathSegmentor, the first text-prompted segmentation foundation model designed specifically for pathology images. We also introduce PathSeg, the largest and most comprehensive dataset for pathology segmentation, built from 21 public sources and containing 275k image-mask-label triples across 160 diverse categories. With PathSegmentor, users can perform semantic segmentation using natural language prompts, eliminating the need for laborious spatial inputs such as points or boxes. Extensive experiments demonstrate that PathSegmentor outperforms specialized models with higher accuracy and broader applicability, while maintaining a compact architecture. It significantly surpasses existing spatial- and text-prompted models by 0.145 and 0.429 in overall Dice scores, respectively, showing strong robustness in segmenting complex structures and generalizing to external datasets. Moreover, PathSegmentor's outputs enhance the interpretability of diagnostic models through feature importance estimation and imaging biomarker discovery, offering pathologists evidence-based support for clinical decision-making. This work advances the development of explainable AI in precision oncology.

Related papers

• VISTA-PATH: An interactive foundation model for pathology image segmentation and quantitative analysis in computational pathology [12.972784296124756]
  VISTA-PATH is an interactive, class-aware pathology segmentation foundation model.<n>It produces pixel-level segmentation that are directly meaningful for clinical interpretation.<n>We show that VISTA-PATH is a preferred model for computational pathology.
  arXiv  Detail & Related papers  (2026-01-23T05:06:57Z)
• Multi-Sequence Parotid Gland Lesion Segmentation via Expert Text-Guided Segment Anything Model [10.892625199593644]
  The parotid gland segment anything model (PG-SAM) incorporates expert domain knowledge for cross-sequence parotid gland lesion segmentation.<n>The PG-SAM achieves state-of-the-art performance in parotid gland lesion segmentation across three independent clinical centers.
  arXiv  Detail & Related papers  (2025-08-13T09:25:29Z)
• CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography [27.1055374364626]
  We present CADS, an open-source framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous data sources for whole-body CT segmentation.<n>At its core is a large-scale dataset of 22,022 CT volumes with complete annotations for 167 anatomical structures.<n>Through comprehensive evaluation across 18 public datasets and an independent real-world hospital cohort, we demonstrate advantages over SoTA approaches.
  arXiv  Detail & Related papers  (2025-07-29T19:58:32Z)
• PathSegDiff: Pathology Segmentation using Diffusion model representations [63.20694440934692]
  We propose PathSegDiff, a novel approach for histopathology image segmentation that leverages Latent Diffusion Models (LDMs) as pre-trained featured extractors.<n>Our method utilizes a pathology-specific LDM, guided by a self-supervised encoder, to extract rich semantic information from H&E stained histopathology images.<n>Our experiments demonstrate significant improvements over traditional methods on the BCSS and GlaS datasets.
  arXiv  Detail & Related papers  (2025-04-09T14:58:21Z)
• Multi-Class Segmentation of Aortic Branches and Zones in Computed Tomography Angiography: The AortaSeg24 Challenge [55.252714550918824]
  AortaSeg24 MICCAI Challenge introduced the first dataset of 100 CTA volumes annotated for 23 clinically relevant aortic branches and zones.<n>This paper presents the challenge design, dataset details, evaluation metrics, and an in-depth analysis of the top-performing algorithms.
  arXiv  Detail & Related papers  (2025-02-07T21:09:05Z)
• Pitfalls of topology-aware image segmentation [81.19923502845441]
  We identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts, and inappropriate use of evaluation metrics.<n>We propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.
  arXiv  Detail & Related papers  (2024-12-19T08:11:42Z)
• MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation [2.2585213273821716]
  We introduce MedCLIP-SAMv2, a novel framework that integrates the CLIP and SAM models to perform segmentation on clinical scans.<n>Our approach includes fine-tuning the BiomedCLIP model with a new Decoupled Hard Negative Noise Contrastive Estimation (DHN-NCE) loss.<n>We also investigate using zero-shot segmentation labels within a weakly supervised paradigm to enhance segmentation quality further.
  arXiv  Detail & Related papers  (2024-09-28T23:10:37Z)
• Scribble-Based Interactive Segmentation of Medical Hyperspectral Images [4.675955891956077]
  This work introduces a scribble-based interactive segmentation framework for medical hyperspectral images. The proposed method utilizes deep learning for feature extraction and a geodesic distance map generated from user-provided scribbles.
  arXiv  Detail & Related papers  (2024-08-05T12:33:07Z)
• Anatomy-guided Pathology Segmentation [56.883822515800205]
  We develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features. Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy. In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods.
  arXiv  Detail & Related papers  (2024-07-08T11:44:15Z)
• Analysing the effectiveness of a generative model for semi-supervised medical image segmentation [23.898954721893855]
  State-of-the-art in automated segmentation remains supervised learning, employing discriminative models such as U-Net. Semi-supervised learning (SSL) attempts to leverage the abundance of unlabelled data to obtain more robust and reliable models. Deep generative models such as the SemanticGAN are truly viable alternatives to tackle challenging medical image segmentation problems.
  arXiv  Detail & Related papers  (2022-11-03T15:19:59Z)
• Spatially Dependent U-Nets: Highly Accurate Architectures for Medical Imaging Segmentation [10.77039660100327]
  We introduce a novel deep neural network architecture that exploits the inherent spatial coherence of anatomical structures. Our approach is well equipped to capture long-range spatial dependencies in the segmented pixel/voxel space. Our method performs favourably to commonly used U-Net and U-Net++ architectures.
  arXiv  Detail & Related papers  (2021-03-22T10:37:20Z)
• Weakly supervised multiple instance learning histopathological tumor segmentation [51.085268272912415]
  We propose a weakly supervised framework for whole slide imaging segmentation. We exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset.
  arXiv  Detail & Related papers  (2020-04-10T13:12:47Z)
• VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images [121.31355003451152]
  Large Scale Vertebrae Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020. We present the the results of this evaluation and further investigate the performance-variation at vertebra-level, scan-level, and at different fields-of-view.
  arXiv  Detail & Related papers  (2020-01-24T21:09:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.