Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images

• URL: http://arxiv.org/abs/2507.13974v1
• Date: Fri, 18 Jul 2025 14:38:25 GMT
• Title: Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images
• Authors: Jiaqi Lv, Yijie Zhu, Carmen Guadalupe Colin Tenorio, Brinder Singh Chohan, Mark Eastwood, Shan E Ahmed Raza,
• Abstract summary: We propose a novel deep learning network for the segmentation of five tissue classes in melanoma H&E images.<n>Our approach leverages Virchow2, a pathology foundation model trained on 3.1 million histopathology images as a feature extractor.<n>The proposed model achieved first place in the tissue segmentation task of the PUMA Grand Challenge, demonstrating robust performance and generalizability.
• Score: 4.058897726957504
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Melanoma is an aggressive form of skin cancer with rapid progression and high metastatic potential. Accurate characterisation of tissue morphology in melanoma is crucial for prognosis and treatment planning. However, manual segmentation of tissue regions from haematoxylin and eosin (H&E) stained whole-slide images (WSIs) is labour-intensive and prone to inter-observer variability, this motivates the need for reliable automated tissue segmentation methods. In this study, we propose a novel deep learning network for the segmentation of five tissue classes in melanoma H&E images. Our approach leverages Virchow2, a pathology foundation model trained on 3.1 million histopathology images as a feature extractor. These features are fused with the original RGB images and subsequently processed by an encoder-decoder segmentation network (Efficient-UNet) to produce accurate segmentation maps. The proposed model achieved first place in the tissue segmentation task of the PUMA Grand Challenge, demonstrating robust performance and generalizability. Our results show the potential and efficacy of incorporating pathology foundation models into segmentation networks to accelerate computational pathology workflows.

Related papers

• CytoFM: The first cytology foundation model [3.591868126270513]
  We introduce CytoFM, the first self-supervised foundation model for digital Cytology.<n>We pretrain CytoFM on a diverse collection of datasets to learn robust, transferable representations.<n>Our results demonstrate that CytoFM performs better on two out of three downstream tasks than existing foundation models pretrained on histopathology.
  arXiv  Detail & Related papers  (2025-04-18T01:37:50Z)
• PathSegDiff: Pathology Segmentation using Diffusion model representations [63.20694440934692]
  We propose PathSegDiff, a novel approach for histopathology image segmentation that leverages Latent Diffusion Models (LDMs) as pre-trained featured extractors.<n>Our method utilizes a pathology-specific LDM, guided by a self-supervised encoder, to extract rich semantic information from H&E stained histopathology images.<n>Our experiments demonstrate significant improvements over traditional methods on the BCSS and GlaS datasets.
  arXiv  Detail & Related papers  (2025-04-09T14:58:21Z)
• GS-TransUNet: Integrated 2D Gaussian Splatting and Transformer UNet for Accurate Skin Lesion Analysis [44.99833362998488]
  We present a novel approach that combines 2D Gaussian splatting with the Transformer UNet architecture for automated skin cancer diagnosis.<n>Our findings illustrate significant advancements in the precision of segmentation and classification.<n>This integration sets new benchmarks in the field and highlights the potential for further research into multi-task medical image analysis methodologies.
  arXiv  Detail & Related papers  (2025-02-23T23:28:47Z)
• Adapting Segment Anything Model to Melanoma Segmentation in Microscopy Slide Images [0.1534667887016089]
  We present a novel approach that uses the Segment Anything Model (SAM) for automatic melanoma segmentation in microscopy slide images. Our method employs an initial semantic segmentation model to generate preliminary segmentation masks that are then used to prompt SAM. Our experimental results demonstrate that this approach significantly outperforms the baseline Segformer by 9.1% in terms of IoU.
  arXiv  Detail & Related papers  (2024-10-03T04:40:18Z)
• ShapeMamba-EM: Fine-Tuning Foundation Model with Local Shape Descriptors and Mamba Blocks for 3D EM Image Segmentation [49.42525661521625]
  This paper presents ShapeMamba-EM, a specialized fine-tuning method for 3D EM segmentation. It is tested over a wide range of EM images, covering five segmentation tasks and 10 datasets.
  arXiv  Detail & Related papers  (2024-08-26T08:59:22Z)
• Deep learning-based instance segmentation for the precise automated quantification of digital breast cancer immunohistochemistry images [1.8434042562191815]
  We demonstrate the feasibility of using a deep learning-based instance segmentation architecture for the automatic quantification of both nuclear and membrane biomarkers applied to IHC-stained slides. We have collected annotations over samples of HE, ER and Ki-67 (nuclear biomarkers) and HER2 (membrane biomarker) IHC-stained images. We have trained two models, so-called nuclei- and membrane-aware segmentation models, which, once successfully validated, have revealed to be a promising method to segment nuclei instances in IHC-stained images.
  arXiv  Detail & Related papers  (2023-11-22T22:23:47Z)
• Histopathologic Cancer Detection [0.0]
  This work uses the PatchCamelyon benchmark datasets and trains them in a multi-layer perceptron and convolution model to observe the model's performance in terms of precision Recall, F1 Score, Accuracy, and AUC Score. Also, this paper introduced ResNet50 and InceptionNet models with data augmentation, where ResNet50 is able to beat the state-of-the-art model.
  arXiv  Detail & Related papers  (2023-11-13T19:51:46Z)
• Medulloblastoma Tumor Classification using Deep Transfer Learning with Multi-Scale EfficientNets [63.62764375279861]
  We propose an end-to-end MB tumor classification and explore transfer learning with various input sizes and matching network dimensions. Using a data set with 161 cases, we demonstrate that pre-trained EfficientNets with larger input resolutions lead to significant performance improvements.
  arXiv  Detail & Related papers  (2021-09-10T13:07:11Z)
• Spectral-Spatial Recurrent-Convolutional Networks for In-Vivo Hyperspectral Tumor Type Classification [49.32653090178743]
  We demonstrate the feasibility of in-vivo tumor type classification using hyperspectral imaging and deep learning. Our best model achieves an AUC of 76.3%, significantly outperforming previous conventional and deep learning methods.
  arXiv  Detail & Related papers  (2020-07-02T12:00:53Z)
• Gleason Grading of Histology Prostate Images through Semantic Segmentation via Residual U-Net [60.145440290349796]
  The final diagnosis of prostate cancer is based on the visual detection of Gleason patterns in prostate biopsy by pathologists. Computer-aided-diagnosis systems allow to delineate and classify the cancerous patterns in the tissue. The methodological core of this work is a U-Net convolutional neural network for image segmentation modified with residual blocks able to segment cancerous tissue.
  arXiv  Detail & Related papers  (2020-05-22T19:49:10Z)
• Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images [152.34988415258988]
  Automated detection of lung infections from computed tomography (CT) images offers a great potential to augment the traditional healthcare strategy for tackling COVID-19. segmenting infected regions from CT slices faces several challenges, including high variation in infection characteristics, and low intensity contrast between infections and normal tissues. To address these challenges, a novel COVID-19 Deep Lung Infection Network (Inf-Net) is proposed to automatically identify infected regions from chest CT slices.
  arXiv  Detail & Related papers  (2020-04-22T07:30:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.