Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

• URL: http://arxiv.org/abs/2604.23314v1
• Date: Sat, 25 Apr 2026 14:09:11 GMT
• Title: Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
• Abstract summary: Segment Anything Model (SAM) offers powerful zero-shot capabilities, but collapses under the weak, generic, and noisy prompts that dominate real clinicals.<n>We introduce SPD, a Saliency-Guided Prompt Distillation framework that converts unreliable cues into robust guidance.<n>SPD consistently outperforms existing SAM adaptations and delivers large gains in both region-based and boundary-based metrics.
• Score: 46.344226429758606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Segmentation is central to clinical diagnosis and monitoring, yet the reliability of modern foundation models in medical imaging still depends on the availability of precise prompts. The Segment Anything Model (SAM) offers powerful zero-shot capabilities, although it collapses under the weak, generic, and noisy prompts that dominate real clinical workflows. In practice, annotations such as centerline points are coarse and ambiguous, often drifting across neighboring anatomy and misguiding SAM toward inconsistent or incomplete masks. We introduce SPD, a Saliency-Guided Prompt Distillation framework that converts these unreliable cues into robust guidance. SPD first learns data-driven anatomical priors through a lightweight saliency head to obtain confident localization maps. These priors then drive Contextual Prompt Distillation, which validates and enriches noisy prompts using cues from anatomically adjacent slices, producing a consensus prompt set that matches the behavior of expert reasoning. A Pairwise Slice Consistency objective further enforces local anatomical coherence during segmentation. Experiments on four challenging MRI and CT benchmarks demonstrate that SPD consistently outperforms existing SAM adaptations and supervised baselines, delivering large gains in both region-based and boundary-based metrics. SPD provides a practical and principled path toward reliable foundation model deployment in clinical environments where only imperfect prompts are available.

Related papers

• Prompt-Guided Interactive Segmentation of Interstitial Lung Disease in Thoracic CT [0.0]
  We present the first adaptation of MedSAM2 for interactive 3D ILD segmentation on thoracic CT.<n>We investigate three fine-tuning strategies and multiple clinically motivated prompts: bounding-boxes (BBox), point, lasso, and scribble.<n>On a dataset spanning seven ILD patterns and healthy lung tissue, full model fine-tuning performed best.<n>While BBox prompts achieve the strongest performance, non-native MedSAM2 interactions such as lasso and scribble prompts also prove effective.
  arXiv  Detail & Related papers  (2026-08-28T15:43:25Z)
• Counterfactual Anatomy-guided Spatial-Temporal Decoding for Annotation-Free Hallucination Mitigation in Medical VLMs [21.705626227651624]
  We propose Counterfactual Anatomy-guided Spatial-Temporal decoding (CAST)<n>CAST operates entirely during inference and requires no manual annotations for anatomically grounded hallucination mitigation.<n>CAST consistently outperforms strong baselines and surpasses decoding strategies reliant on ground truth.
  arXiv  Detail & Related papers  (2026-08-18T06:45:08Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• Progressive Uncertainty-Guided Evidential U-KAN for Trustworthy Medical Image Segmentation [11.953394478206581]
  We propose a progressive evidence uncertainty guided attention (PEUA) mechanism to guide the model to focus on the feature representation learning of hard regions.<n>Unlike conventional approaches, PEUA progressively refines attention using uncertainty maps while employing low-rank learning to denoise attention weights.<n>We thus introduce a semantic-preserving evidence learning (SAEL) strategy, integrating a semantic-smooth evidence generator and a fidelity-enhancing regularization term to retain critical semantics.
  arXiv  Detail & Related papers  (2025-10-10T02:58:01Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• Human-Guided Shade Artifact Suppression in CBCT-to-MDCT Translation via Schrödinger Bridge with Conditional Diffusion [1.5869861104370917]
  We present a novel framework for CBCT-to-MDCT translation, grounded in the Schrodinger Bridge (SB) formulation.<n>Our approach explicitly enforces boundary consistency between CBCT inputs and pseudo targets, ensuring both anatomical fidelity and perceptual controllability.
  arXiv  Detail & Related papers  (2025-07-15T06:44:53Z)
• Point-supervised Brain Tumor Segmentation with Box-prompted MedSAM [11.861051009791224]
  We introduce an iterative framework to facilitate semantic-aware point-supervised MedSAM. Box-prompt generator (SBPG) module has the capacity to convert the point input into potential pseudo bounding box suggestions. Prompt-guided spatial refinement (PGSR) module harnesses the exceptional generalizability of MedSAM to infer the segmentation mask.
  arXiv  Detail & Related papers  (2024-08-01T16:52:39Z)
• SAM-Driven Weakly Supervised Nodule Segmentation with Uncertainty-Aware Cross Teaching [13.5553526185399]
  Automated nodule segmentation is essential for computer-assisted diagnosis in ultrasound images. Recently, segmentation foundation models like SAM have shown impressive generalizability on natural images. In this work, we devise a novel weakly supervised framework that effectively utilizes the segmentation foundation model to generate pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-18T14:27:54Z)
• CT-based brain ventricle segmentation via diffusion Schrödinger Bridge without target domain ground truths [0.9720086191214947]
  Efficient and accurate brain ventricle segmentation from clinical CT scans is critical for emergency surgeries like ventriculostomy. We introduce a novel uncertainty-aware ventricle segmentation technique without the need of CT segmentation ground truths. Our method employs the diffusion Schr"odinger Bridge and an attention recurrent residual U-Net to capitalize on unpaired CT and MRI scans.
  arXiv  Detail & Related papers  (2024-05-28T15:17:58Z)
• Real-time guidewire tracking and segmentation in intraoperative x-ray [52.51797358201872]
  We propose a two-stage deep learning framework for real-time guidewire segmentation and tracking. In the first stage, a Yolov5 detector is trained, using the original X-ray images as well as synthetic ones, to output the bounding boxes of possible target guidewires. In the second stage, a novel and efficient network is proposed to segment the guidewire in each detected bounding box.
  arXiv  Detail & Related papers  (2024-04-12T20:39:19Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• On the Practicality of Deterministic Epistemic Uncertainty [106.06571981780591]
  deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution data. It remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications.
  arXiv  Detail & Related papers  (2021-07-01T17:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.