Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts

• URL: http://arxiv.org/abs/2603.21083v1
• Date: Sun, 22 Mar 2026 06:45:02 GMT
• Title: Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts
• Abstract summary: TextBraTS is a hierarchical text-guided framework that builds on the TextBraTS baseline with three novel components.<n>Sub-region-aware prompt tuning, which uses learnable soft prompts with a LoRA-adapted BioBERT encoder, generates specialized text representations tailored for each sub-region.<n> Experiments on the TextBraTS dataset demonstrate consistent improvements across all sub-regions against state-of-the-art methods by 1.7% and 6% on the main metrics Dice and HD95.
• Score: 50.36510050690281
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Brain tumor segmentation remains challenging because the three standard sub-regions, i.e., whole tumor (WT), tumor core (TC), and enhancing tumor (ET), often exhibit ambiguous visual boundaries. Integrating radiological description texts with imaging has shown promise. However, most multimodal approaches typically compress a report into a single global text embedding shared across all sub-regions, overlooking their distinct clinical characteristics. We propose TextCSP (text-modulated soft cascade architecture), a hierarchical text-guided framework that builds on the TextBraTS baseline with three novel components: (1) a text-modulated soft cascade decoder that predicts WT->TC->ET in a coarse-to-fine manner consistent with their anatomical containment hierarchy. (2) sub-region-aware prompt tuning, which uses learnable soft prompts with a LoRA-adapted BioBERT encoder to generate specialized text representations tailored for each sub-region; (3) text-semantic channel modulators that convert the aforementioned representations into channel-wise refinement signals, enabling the decoder to emphasize features aligned with clinically described patterns. Experiments on the TextBraTS dataset demonstrate consistent improvements across all sub-regions against state-of-the-art methods by 1.7% and 6% on the main metrics Dice and HD95.

Related papers

• Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation [81.21550084361755]
  Language-guided Medical Image (LMIS) has shown great potential to improve the delineation of anatomical structures and lesions.<n> Existing methods rely on either implicit interaction between textual and visual features or auxiliary coarse-grained supervision for cross-modal alignment.<n>We propose Text-as-Illumination Retinex Network (TIRNet), a novel framework that treats text embeddings as semantic illumination for feature modulation.
  arXiv  Detail & Related papers  (2026-06-26T07:27:48Z)
• TGC-Net: A Structure-Aware and Semantically-Aligned Framework for Text-Guided Medical Image Segmentation [56.09179939570486]
  We propose TGC-Net, a CLIP-based framework focusing on parameter-efficient, task-specific adaptations.<n>TGC-Net achieves state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-12-24T12:06:26Z)
• XBusNet: Text-Guided Breast Ultrasound Segmentation via Multimodal Vision-Language Learning [0.0]
  XBusNet is a novel dual-prompt, dual-branch multimodal model that combines image features with clinically grounded text.<n>We evaluate it on the Breast Lesions USG (BLU) dataset using five-fold cross-validation.<n>Small lesions show the largest gains, with fewer missed regions and fewer spurious activations.
  arXiv  Detail & Related papers  (2025-09-08T20:45:55Z)
• MedRegion-CT: Region-Focused Multimodal LLM for Comprehensive 3D CT Report Generation [1.6515663221123749]
  We propose MedRegion-CT, a region-focused Multi-Modal Large Language Model (MLLM) framework, featuring three key innovations.<n>First, we introduce Region Representative ($R2$) Token Pooling, which utilizes a 2D-wise pretrained vision model to efficiently extract 3D CT features.<n>Second, a universal segmentation model generates pseudo-masks, which are then processed by a mask encoder to extract region-centric features.<n>Third, we leverage segmentation results to extract patient-specific attributions, including organ size, diameter, and locations.
  arXiv  Detail & Related papers  (2025-06-29T06:08:55Z)
• Large Language Model with Region-guided Referring and Grounding for CT Report Generation [4.804660464589285]
  Existing methods primarily only consider the global features of the entire volume.<n>We propose Reg2RG, the first region-guided referring and grounding framework for CT report generation.
  arXiv  Detail & Related papers  (2024-11-23T12:25:06Z)
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis [56.57177181778517]
  RadGenome-Chest CT is a large-scale, region-guided 3D chest CT interpretation dataset based on CT-RATE. We leverage the latest powerful universal segmentation and large language models to extend the original datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:11:37Z)
• GuideGen: A Text-Guided Framework for Full-torso Anatomy and CT Volume Generation [1.138481191622247]
  GuideGen is a controllable framework that generates anatomical masks and corresponding CT volumes for the entire torso-from chest to pelvis-based on free-form text prompts. Our approach includes three core components: a text-conditional semantic synthesizer for creating realistic full-torso anatomies; a contrast-aware autoencoder for detailed, high-fidelity feature extraction across varying contrast levels; and a latent feature generator that ensures alignment between CT images, anatomical semantics and input prompts.
  arXiv  Detail & Related papers  (2024-03-12T02:09:39Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• BCS-Net: Boundary, Context and Semantic for Automatic COVID-19 Lung Infection Segmentation from CT Images [83.82141604007899]
  BCS-Net is a novel network for automatic COVID-19 lung infection segmentation from CT images. BCS-Net follows an encoder-decoder architecture, and more designs focus on the decoder stage. In each BCSR block, the attention-guided global context (AGGC) module is designed to learn the most valuable encoder features for decoder.
  arXiv  Detail & Related papers  (2022-07-17T08:54:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.