Exploring Prompt Alignment with Clinical Factors in Zero-Shot Segmentation VLMs for NSCLC Tumor Segmentation

• URL: http://arxiv.org/abs/2605.01266v1
• Date: Sat, 02 May 2026 05:49:32 GMT
• Title: Exploring Prompt Alignment with Clinical Factors in Zero-Shot Segmentation VLMs for NSCLC Tumor Segmentation
• Abstract summary: Zero-shot vision-language models (VLMs) offer a promptable alternative to task-specific training for gross tumor volume delineation.<n>We study alignment directions in VoxTell on a held-out internal NSCLC tumor dataset.<n> Alignment analyses revealed that anatomical location is the dominant driver of VoxTell's spatial attention.
• Score: 0.05597620745943382
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-shot vision-language models (VLMs) offer a promptable alternative to task-specific training for gross tumor volume (GTV) delineation in non-small-cell lung cancer (NSCLC), but the prompt dimensions that govern their spatial behavior remain poorly understood. We study this question by probing alignment directions in VoxTell on a held-out internal NSCLC tumor dataset through sub-prompt decomposition into diagnosis, demographic, staging, anatomical, generic, and irrelevant controls; attribute-wise perturbation robustness; specificity ladders; and cross-case prompt swaps, while benchmarking against fine-tuned and zero-shot baselines using the Dice Similarity Coefficient (DSC) with Wilcoxon signed-rank tests and Benjamini-Hochberg correction. Alignment analyses revealed that anatomical location is the dominant driver of VoxTell's spatial attention: 63.4 percent of location perturbations caused catastrophic drops, prompt specificity improved from generic to full descriptions except for diagnosis-only prompts, irrelevant prompts correctly yielded zero segmentation, and cross-case prompt swaps confirmed patient-specific conditioning (matched DSC 0.906 vs. mismatched 0.406). Histology and stage substitutions had minimal effect, indicating that the model prioritizes "where to look" over "what to look for." In this context, VoxTell, operating fully zero-shot, achieved a mean DSC of 0.613, statistically indistinguishable from nnUNet (0.690, adjusted p = 0.156) and Ahmed et al. (0.675, adjusted p = 0.679), while significantly outperforming all other zero-shot models. Together, these findings argue that segmentation VLMs should be evaluated not only by Dice, but also by the prompt dimensions to which they align.

Related papers

• Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment [29.08356862626618]
  Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference.<n>We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model.
  arXiv  Detail & Related papers  (2026-08-30T11:49:54Z)
• Post-Operative Glioma Segmentation via Loss Stabilization, Normalization and Subspace Attention [46.03321798937855]
  We show that the standard Generalized Dice Loss (GDL) is unstable under domain shift.<n>We propose a Subspace-Aware Class Attention (SACA) module that re-calibrates the bottleneck features and raises Enhancing Tumor (ET) sensitivity by 8%.
  arXiv  Detail & Related papers  (2026-07-23T09:59:30Z)
• SVC-Probe: A Framework for Evaluating Perturbation Generalization in Spatial Foundation-Model Embeddings [3.884856136722027]
  This work examines generalization in spatial foundation-model embeddings derived from fluorescence microscopy images.<n>We introduce SVC-Probe, a perturbation-aware framework that combines Subcellular Embedding Atlas Stability, Mondrian Neighborhood Graphs, and a Foundation Model Perturbation Probe.
  arXiv  Detail & Related papers  (2026-06-26T13:46:36Z)
• SegGuidedNet: Sub-Region-Aware Attention Supervision for Interpretable Brain Tumor Segmentation [5.995766869240907]
  We propose SegGuidedNet, a decoder network that supervises the decoder to produce spatially discriminative attention maps for each tumour sub-region necrotic core, peritumoral oedema, and enhancing tumour via a lightweight auxiliary loss, adding less than 0.2% parameter overhead.<n>SegGuidedNet achieves mean Dice of 0.905, surpassing ensemble-based nnU-Net and HNF-Netv2 as a single model and approaching Swin UNETR a 10-model ensemble within 2--4 Dice points at a fraction of the inference cost.
  arXiv  Detail & Related papers  (2026-05-21T14:50:04Z)
• MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors [3.346370180968566]
  We introduce MedDialBench, a benchmark enabling controlled, dose-response characterization of how individual patient behavior dimensions affect diagnostic robustness.<n>It decomposes patient behavior into five dimensions -- Logic Consistency, Health Cognition, Expression Style, Disclosure, and Attitude -- each with graded severity levels and case-specific behavioral scripts.<n>This controlled factorial design enables graded sensitivity analysis, dose-response profiling, and cross-dimension interaction detection.
  arXiv  Detail & Related papers  (2026-04-08T09:09:08Z)
• A Unified Framework for Joint Detection of Lacunes and Enlarged Perivascular Spaces [3.9313804276175506]
  Cerebral small vessel disease (CSVD) markers, specifically enlarged perivascular spaces (EPVS) and lacunae, present a unique challenge in medical image analysis.<n>We propose a morphologydecoupled framework where Zero- Gated CrossTask Attention exploits dense EPVS context to guide sparse lacune detection.
  arXiv  Detail & Related papers  (2026-03-04T16:30:46Z)
• A Detection-Gated Pipeline for Robust Glottal Area Waveform Extraction and Clinical Pathology Assessment [0.0]
  We propose a detection-gated pipeline that integrates a localizer with a segmenter.<n>The pipeline achieved state-of-the-art consistency on the GIRAFE and BAGLS benchmarks.
  arXiv  Detail & Related papers  (2026-03-02T17:05:41Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• Hide-and-Seek Attribution: Weakly Supervised Segmentation of Vertebral Metastases in CT [68.09387763135236]
  We introduce a weakly supervised method trained solely on vertebra-level healthy/malignant labels, without any lesion masks.<n>We achieve strong blastic/lytic performance despite no mask supervision.
  arXiv  Detail & Related papers  (2025-12-07T14:03:28Z)
• MeCaMIL: Causality-Aware Multiple Instance Learning for Fair and Interpretable Whole Slide Image Diagnosis [40.3028468133626]
  Multiple instance learning (MIL) has emerged as the dominant paradigm for whole slide image (WSI) analysis in computational pathology.<n>textbfMeCaMIL, a causality-aware MIL framework, explicitly models demographic confounders through structured causal graphs.<n>MeCaMIL achieves superior fairness -- demographic disparity variance drops by over 65% relative reduction on average across attributes.
  arXiv  Detail & Related papers  (2025-11-14T06:47:21Z)
• Conformal Lesion Segmentation for 3D Medical Images [82.92159832699583]
  We propose a risk-constrained framework that calibrates data-driven thresholds via conformalization to ensure the test-time FNR remains below a target tolerance.<n>We validate the statistical soundness and predictive performance of CLS on six 3D-LS datasets across five backbone models, and conclude with actionable insights for deploying risk-aware segmentation in clinical practice.
  arXiv  Detail & Related papers  (2025-10-19T08:21:00Z)
• Depth-Sequence Transformer (DST) for Segment-Specific ICA Calcification Mapping on Non-Contrast CT [38.85617601239779]
  Conventional 3D models are forced to process downsampled volumes or isolated patches.<n>We reformulate the 3D challenge as a textbfParallel Probabilistic Landmark localization task along the 1D axial dimension.<n>We propose the textbfDepth-Sequence Transformer (DST), a framework that processes full-resolution CT volumes as sequences of 2D slices.
  arXiv  Detail & Related papers  (2025-07-10T23:12:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.