SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation

• URL: http://arxiv.org/abs/2507.23673v1
• Date: Thu, 31 Jul 2025 15:49:57 GMT
• Title: SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation
• Authors: Alfie Roddan, Tobias Czempiel, Chi Xu, Daniel S. Elson, Stamatia Giannarou,
• Abstract summary: We introduce SAMSA, a novel interactive segmentation framework that combines an RGB foundation model with spectral analysis.<n>Performance evaluation on publicly available datasets has shown 81.0% 1-click and 93.4% 5-click DICE.<n>Our approach enables seamless integration of datasets with different spectral characteristics, providing a flexible framework for hyperspectral medical image analysis.
• Score: 5.9804425273111095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperspectral imaging (HSI) provides rich spectral information for medical imaging, yet encounters significant challenges due to data limitations and hardware variations. We introduce SAMSA, a novel interactive segmentation framework that combines an RGB foundation model with spectral analysis. SAMSA efficiently utilizes user clicks to guide both RGB segmentation and spectral similarity computations. The method addresses key limitations in HSI segmentation through a unique spectral feature fusion strategy that operates independently of spectral band count and resolution. Performance evaluation on publicly available datasets has shown 81.0% 1-click and 93.4% 5-click DICE on a neurosurgical and 81.1% 1-click and 89.2% 5-click DICE on an intraoperative porcine hyperspectral dataset. Experimental results demonstrate SAMSA's effectiveness in few-shot and zero-shot learning scenarios and using minimal training examples. Our approach enables seamless integration of datasets with different spectral characteristics, providing a flexible framework for hyperspectral medical image analysis.

Related papers

• SAMSA 2.0: Prompting Segment Anything with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation [5.9804425273111095]
  We present SAMSA 2.0, an interactive segmentation framework for hyperspectral medical imaging.<n>It introduces spectral angle prompting to guide the Segment Anything Model (SAM) using spectral similarity alongside spatial cues.<n>Without retraining, SAMSA 2.0 achieves up to +3.8% higher Dice scores compared to RGB-only models and up to +3.1% over prior spectral fusion methods.
  arXiv  Detail & Related papers  (2025-08-01T10:16:26Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [76.95536611263356]
  PolSAR data presents unique challenges due to its rich and complex characteristics.<n>Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used.<n>Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively.<n>We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• A Hyperspectral Imaging Dataset and Methodology for Intraoperative Pixel-Wise Classification of Metastatic Colon Cancer in the Liver [0.0]
  Hyperspectral imaging holds significant potential for transforming the field of computational pathology. There is currently a shortage of pixel-wise annotated HSI data necessary for training deep learning (DL) models. We present a database consisted of 27 HSIs of hematoxylin-eosin stained frozen sections, collected from 14 patients with colon adenocarcinoma metastasized to the liver.
  arXiv  Detail & Related papers  (2024-11-11T13:17:55Z)
• Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development [59.74920439478643]
  In this paper, we collect and annotated the first benchmark dataset that covers diverse ERUS scenarios. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. We introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR)
  arXiv  Detail & Related papers  (2024-08-19T15:04:42Z)
• Scribble-Based Interactive Segmentation of Medical Hyperspectral Images [4.675955891956077]
  This work introduces a scribble-based interactive segmentation framework for medical hyperspectral images. The proposed method utilizes deep learning for feature extraction and a geodesic distance map generated from user-provided scribbles.
  arXiv  Detail & Related papers  (2024-08-05T12:33:07Z)
• TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI [59.86827659781022]
  A nnU-Net model (TotalSegmentator) was trained on MRI and segment 80atomic structures.<n>Dice scores were calculated between the predicted segmentations and expert reference standard segmentations to evaluate model performance.<n>Open-source, easy-to-use model allows for automatic, robust segmentation of 80 structures.
  arXiv  Detail & Related papers  (2024-05-29T20:15:54Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation [55.9217962930169]
  We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images. S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
  arXiv  Detail & Related papers  (2023-06-14T09:01:50Z)
• Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines [0.13854111346209866]
  Segment Anything Model (SAM) harnesses a massive training dataset to segment nearly any object. Our findings reveal that SAM's zero-shot performance is not only comparable, but in certain cases, surpasses the current state-of-the-art. We propose practical guidelines that require minimal interaction while consistently yielding robust outcomes.
  arXiv  Detail & Related papers  (2023-04-28T22:07:24Z)
• Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation [5.547422331445511]
  We report quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks. Our study indicates the versatility of generalist vision foundation models on medical imaging.
  arXiv  Detail & Related papers  (2023-04-25T08:07:59Z)
• Hyperspectral Image Segmentation based on Graph Processing over Multilayer Networks [51.15952040322895]
  One important task of hyperspectral image (HSI) processing is the extraction of spectral-spatial features. We propose several approaches to HSI segmentation based on M-GSP feature extraction. Our experimental results demonstrate the strength of M-GSP in HSI processing and spectral-spatial information extraction.
  arXiv  Detail & Related papers  (2021-11-29T23:28:18Z)
• Robust deep learning-based semantic organ segmentation in hyperspectral images [29.342448910787773]
  Full-scene semantic segmentation based on spectral imaging data and obtained during open surgery has received almost no attention to date. We are investigating the following research questions based on hyperspectral imaging (HSI) data of pigs acquired in an open surgery setting. We conclude that HSI could become a powerful image modality for fully-automatic surgical scene understanding.
  arXiv  Detail & Related papers  (2021-11-09T20:37:38Z)
• Cross-Spectral Periocular Recognition with Conditional Adversarial Networks [59.17685450892182]
  We propose Conditional Generative Adversarial Networks, trained to con-vert periocular images between visible and near-infrared spectra. We obtain a cross-spectral periocular performance of EER=1%, and GAR>99% @ FAR=1%, which is comparable to the state-of-the-art with the PolyU database.
  arXiv  Detail & Related papers  (2020-08-26T15:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.