Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model

• URL: http://arxiv.org/abs/2608.05389v1
• Date: Wed, 05 Aug 2026 20:17:02 GMT
• Title: Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model
• Abstract summary: 3D vision-language foundation model can perform instruction-guided refinement of glioma subregion segmentations.<n> sensitivity to correct, blank, and contradictory prompts suggests text-dependent contour editing rather than nonspecific post-processing.
• Score: 5.368430154049423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Accurate glioma subregion delineation is important for radiotherapy planning and longitudinal monitoring, but manual contour correction is time-consuming. Models such as nnU-Net may generalize imperfectly and lack clinician-directed text correction. Purpose: We investigated adapting a three-dimensional (3D) vision-language foundation model for text-guided brain tumor segmentation refinement. Methods: We developed a lightweight VoxTell-based framework. Pretrained VoxTell generated initial masks. Oracle prompts derived from segmentation errors encoded target, action, location, imaging evidence, edit size, and preservation constraints. Frozen Qwen/VoxTell prompt embeddings were injected through trainable projections into its multiscale decoder conditioning; other weights remained frozen. Training, validation, and testing used 901, 100, and 250 BraTS-GLI cases. Cross-dataset transfer was evaluated on 100 meningioma, metastasis, pediatric tumor, and UPENN-GBM cases. Results: On the internal test set using post-contrast T1-weighted input, correct instructions improved subregion Dice similarity coefficient (DSC; enhancing tumor, edema, and necrotic/non-enhancing core) from $0.774\pm0.158$ to $0.796\pm0.137$. They outperformed blank prompts ($0.762\pm0.155$; Holm-adjusted $p<0.001$, $d_z=0.71$) and contradictory prompts ($0.770\pm0.163$; $p<0.001$, $d_z=0.48$). In cross-dataset testing, correct instructions improved DSC from $0.527\pm0.287$ to $0.550\pm0.278$ and outperformed contradictory instructions ($0.504\pm0.275$; $p<0.001$, $d_z=0.43$). Conclusion: A 3D vision-language foundation model can perform instruction-guided refinement of glioma subregion segmentations. Sensitivity to correct, blank, and contradictory prompts suggests text-dependent contour editing rather than nonspecific post-processing, supporting further evaluation as a clinician-in-the-loop tool.

Related papers

• Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence [45.78409219344429]
  Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations undermine trustworthy use.<n>We propose a holistic, training-free evidence-injection framework that systematically mitigates hallucinations through dual-side evidence injection.<n>We conduct systematic evaluations on 2 tasks and 5 datasets using textttLLaVA-1.5-7B, texttLLaVA-Med-1.5-7B, texttQwen3-VL-8B/32B, and textttInternVL
  arXiv  Detail & Related papers  (2026-06-30T08:07:25Z)
• Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair [44.80087038178069]
  We introduce Teleological Reasoning Infilling (TRI), a training framework that endows decoder-only transformers with a native emphgoal-conditioned bridging capability.<n>At inference, TRI operates as a surgical repair module within a dual-system loop.<n>Experiments on three benchmarks demonstrate that TRI achieves state-of-the-art performance across all tasks, while reducing per-problem token expenditure by 31.2%.
  arXiv  Detail & Related papers  (2026-06-03T15:58:48Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance [16.566888073449714]
  Vision--language models (VLMs) for radiology report generation (RRG) can produce long-form chest CT reports from volumetric scans.<n>Existing methods face two key limitations: (i) training supervision is often coarse, aligning a whole CT volume with a full free-text report without explicit alignment for fine-grained attributes or pathology locations.<n>We propose emphDiscrimi Cue-Prompting with Prompt Dropout (DCP-PD), a plug-and-play framework that distills fine-grained cues from free-text reports and uses them to guide report generation.
  arXiv  Detail & Related papers  (2026-04-12T03:25:41Z)
• DINOv3 with Test-Time Calibration for Automated Carotid Intima-Media Thickness Measurement on CUBS v1 [5.343632486602401]
  We develop a DINOv3-based framework for carotid intima-media complex segmentation and subsequent CIMT measurement.<n>Our results place a DINOv3-based approach within the clinically relevant $sim$0.1 mm measurement regime.<n>Together, our findings support the feasibility of using vision foundation models for interpretable, calibration-aware CIMT measurement.
  arXiv  Detail & Related papers  (2026-03-11T03:33:04Z)
• R$^{2}$Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection [45.33258156568741]
  Foundation models for medical image segmentation struggle under out-of-distribution shifts.<n>We introduce R$2$Seg, a training-free framework for robust OOD tumor segmentation.
  arXiv  Detail & Related papers  (2025-11-16T17:15:52Z)
• Regional Attention-Enhanced Swin Transformer for Clinically Relevant Medical Image Captioning [4.754208048304996]
  We present a Swin-BART encoder-decoder system with a lightweight regional attention module.<n>Our model achieves state-of-the-art semantic fidelity while remaining compact and interpretable.<n>The proposed design yields accurate, clinically phrased captions and transparent regional attributions.
  arXiv  Detail & Related papers  (2025-11-13T02:55:32Z)
• Real-time nonlinear inversion of magnetic resonance elastography with operator learning [0.06797079068199119]
  The oNLI framework enables real-time MRE inversion (30,000x speedup) of elastograms with comparable spatial accuracy to NLI.<n>A structural prior mechanism, analogous to Soft Prior Regularization in the MRE literature, was incorporated to improve spatial accuracy.
  arXiv  Detail & Related papers  (2025-10-03T08:55:40Z)
• Depth-Sequence Transformer (DST) for Segment-Specific ICA Calcification Mapping on Non-Contrast CT [38.85617601239779]
  Conventional 3D models are forced to process downsampled volumes or isolated patches.<n>We reformulate the 3D challenge as a textbfParallel Probabilistic Landmark localization task along the 1D axial dimension.<n>We propose the textbfDepth-Sequence Transformer (DST), a framework that processes full-resolution CT volumes as sequences of 2D slices.
  arXiv  Detail & Related papers  (2025-07-10T23:12:12Z)
• Advanced Deep Learning Techniques for Automated Segmentation of Type B Aortic Dissections [4.545298205355719]
  We developed four deep learning-based pipelines for Type B aortic dissection segmentation.<n>Our approach achieved superior segmentation accuracy, with Dice Coefficients of 0.91 $pm$ 0.07 for TL, 0.88 $pm$ 0.18 for FL, and 0.47 $pm$ 0.25 for.
  arXiv  Detail & Related papers  (2025-06-27T13:38:33Z)
• Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation [42.06416052431378]
  2D radiology captioning is incompetent to reflect the real-world diagnostic challenge in the volumetric 3D anatomy. We collected an 18,885 text-scan pairs 3D-BrainCT dataset and applied clinical visual instruction tuning to train BrainGPT models to generate radiology-adherent 3D brain CT reports. Our work embodies a holistic framework that showcased the first-hand experience of curating a 3D brain CT dataset, fine-tuning anatomy-sensible language models, and proposing robust radiology evaluation metrics.
  arXiv  Detail & Related papers  (2024-07-02T12:58:35Z)
• CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
  We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning. Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
  arXiv  Detail & Related papers  (2024-04-23T17:59:01Z)
• StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder [48.2010192865749]
  Unsupervised anomaly detection (UAD) can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out of distribution samples. This research proposes a compact version of the "context-encoding" VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA) The proposed pipeline achieved a Dice score of 0.642$pm$0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859$pm$0.112 while detecting artificially induced anomalies.
  arXiv  Detail & Related papers  (2022-01-31T14:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.