UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation

• URL: http://arxiv.org/abs/2607.29200v1
• Date: Fri, 31 Jul 2026 09:19:49 GMT
• Title: UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation
• Abstract summary: UltraSAM3 is a concept-driven foundation model for universal ultrasound image segmentation.<n>It is trained on a large-scale ultrasound segmentation corpus covering 37 public datasets and 13 anatomical categories.<n>Experiments demonstrate that UltraSAM3 consistently outperforms representative concept- and text-driven biomedical segmentation models.
• Score: 22.394292096035603
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ultrasound imaging has become increasingly widespread in clinical practice due to its portability, low cost and real-time capability, making ultrasound image segmentation important. However, ultrasound images differ substantially from CT, MRI, and other medical imaging modalities, as they are often affected by speckle noise, low contrast, acoustic shadows and ambiguous boundaries. Existing ultrasound segmentation methods are still mainly limited to task-specific models or visual-prompt-based foundation models, which are either tailored to particular tasks or require expert-provided visual prompts, making them inconvenient for flexible clinical use. To address these challenges, we propose UltraSAM3, a concept-driven foundation model for universal ultrasound image segmentation. Unlike conventional models, UltraSAM3 enables text-based target specification by adapting SAM3 to ultrasound-specific image--mask--concept triplets. The model is trained on a large-scale ultrasound segmentation corpus covering 37 public datasets and 13 anatomical categories, allowing it to align ultrasound visual patterns with clinically meaningful concepts across diverse organs and lesions. To further improve usability under realistic clinical interaction, we propose an instruction-guided agent that parses complex natural language queries into concise ultrasound concept prompts for UltraSAM3. Extensive experiments demonstrate that UltraSAM3 consistently outperforms representative concept- and text-driven biomedical segmentation models on multi-organ ultrasound benchmarks, external datasets, and visual-prompt-enhanced settings. Moreover, the agent improves segmentation robustness for complex user instructions. These results indicate that ultrasound-specific concept adaptation is effective for building generalizable and interactive ultrasound segmentation foundation models.

Related papers

• SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis [51.81845420301345]
  SonoCLIP is a vision-language foundation model for fetal ultrasound analysis.<n>It integrates segmentation masks as mask-channel visual prompts within the vision encoder.<n>It achieves superior zero-shot transfer performance under both global and mask-guided inference.
  arXiv  Detail & Related papers  (2026-06-28T20:04:49Z)
• Ultrasound-CLIP: Semantic-Aware Contrastive Pre-training for Ultrasound Image-Text Understanding [14.606536256652726]
  Existing vision-language pre-training models, such as CLIP, are primarily designed for other modalities.<n>We construct US-365K, a large-scale ultrasound image-text dataset containing 365k paired samples across 52 anatomical categories.<n>We propose Ultrasound-CLIP, a semantic-aware contrastive learning framework that introduces semantic soft labels and semantic loss to refine sample discrimination.
  arXiv  Detail & Related papers  (2026-04-02T08:10:31Z)
• Exploiting DINOv3-Based Self-Supervised Features for Robust Few-Shot Medical Image Segmentation [3.2564581758935094]
  We propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge.<n>Experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-01-12T23:44:25Z)
• A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications [77.3888788549565]
  We present EchoCare, a novel ultrasound foundation model for generalist clinical use.<n>We developed EchoCare via self-supervised learning on our curated, publicly available, large-scale dataset EchoCareData.<n>With minimal training, EchoCare outperforms state-of-the-art comparison models across 10 representative ultrasound benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T10:05:31Z)
• Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Models [4.981692533734272]
  We propose a prompt-driven vision-language model (VLM) that integrates Grounding DINO with SAM2 to enable object segmentation across multiple ultrasound organs.<n>A total of 18 public ultrasound datasets, encompassing the breast, thyroid, liver, prostate, kidney, and paraspinal muscle, were utilized.<n>Our approach outperforms state-of-the-art segmentation methods, including UniverSeg, MedSAM, MedCLIP-SAM, BiomedParse, and SAMUS.
  arXiv  Detail & Related papers  (2025-06-30T14:33:44Z)
• Semantic Scene Graph for Ultrasound Image Explanation and Scanning Guidance [38.90025585894763]
  This study introduces the scene graph (SG) for ultrasound images to explain image content to ordinary.<n>The predicted SG is explored for its potential in guiding ultrasound scanning toward missing anatomies.<n>The effectiveness of this SG-based image explanation and scanning guidance has been validated on images from the left and right neck regions.
  arXiv  Detail & Related papers  (2025-06-24T14:49:40Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• LOTUS: Learning to Optimize Task-based US representations [39.81131738128329]
  Anatomical segmentation of organs in ultrasound images is essential to many clinical applications. Existing deep neural networks require a large amount of labeled data for training in order to achieve clinically acceptable performance. In this paper, we propose a novel approach for learning to optimize task-based ultra-sound image representations.
  arXiv  Detail & Related papers  (2023-07-29T16:29:39Z)
• OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing [62.993663757843464]
  Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond-duration laser pulses followed by detection of ultrasound waves generated via light-absorption-mediated thermoelastic expansion. OA imaging features a powerful combination between rich optical contrast and high resolution in deep tissues. No standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of OA in clinical settings.
  arXiv  Detail & Related papers  (2022-06-17T08:11:26Z)
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks [48.732863591145964]
  We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
  arXiv  Detail & Related papers  (2021-10-12T21:22:24Z)
• Deep Learning for Ultrasound Beamforming [120.12255978513912]
  Beamforming, the process of mapping received ultrasound echoes to the spatial image domain, lies at the heart of the ultrasound image formation chain. Modern ultrasound imaging leans heavily on innovations in powerful digital receive channel processing. Deep learning methods can play a compelling role in the digital beamforming pipeline.
  arXiv  Detail & Related papers  (2021-09-23T15:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.