LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation

• URL: http://arxiv.org/abs/2603.17576v1
• Date: Wed, 18 Mar 2026 10:33:32 GMT
• Title: LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation
• Abstract summary: We propose LoGSAM, a parameter-efficient framework that transforms radiologist dictation into text prompts for foundation-model-based localization and segmentation.<n>These prompts guide text-conditioned tumor localization via a LoRA-adapted vision-language detection model, Grounding DINO.<n>We evaluate the full pipeline using German dictations from a board-certified radiologist on 12 unseen MRI scans, achieving 91.7% case-level accuracy.
• Score: 5.967422994926725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Precise localization and delineation of brain tumors using Magnetic Resonance Imaging (MRI) are essential for planning therapy and guiding surgical decisions. However, most existing approaches rely on task-specific supervised models and are constrained by the limited availability of annotated data. To address this, we propose LoGSAM, a parameter-efficient, detection-driven framework that transforms radiologist dictation into text prompts for foundation-model-based localization and segmentation. Radiologist speech is first transcribed and translated using a pretrained Whisper ASR model, followed by negation-aware clinical NLP to extract tumor-specific textual prompts. These prompts guide text-conditioned tumor localization via a LoRA-adapted vision-language detection model, Grounding DINO (GDINO). The LoRA adaptation updates using 5% of the model parameters, thereby enabling computationally efficient domain adaptation while preserving pretrained cross-modal knowledge. The predicted bounding boxes are used as prompts for MedSAM to generate pixel-level tumor masks without any additional fine-tuning. Conditioning the frozen MedSAM on LoGSAM-derived priors yields a state-of-the-art dice score of 80.32% on BRISC 2025. In addition, we evaluate the full pipeline using German dictations from a board-certified radiologist on 12 unseen MRI scans, achieving 91.7% case-level accuracy. These results highlight the feasibility of constructing a modular, speech-to-segmentation pipeline by intelligently leveraging pretrained foundation models with minimal parameter updates.

Related papers

• An Automated Radiomics Framework for Postoperative Survival Prediction in Colorectal Liver Metastases using Preoperative MRI [0.8571977422745714]
  We present an automated AI-based framework for postoperative colorectal liver metastasis survival prediction.<n>We developed a survival prediction framework comprising an anatomy-aware segmentation pipeline followed by a radiomics pipeline.<n>Our results show the potential of integrating segmentation algorithms with radiomics-based survival analysis.
  arXiv  Detail & Related papers  (2026-03-10T20:34:24Z)
• Rectified flow-based prediction of post-treatment brain MRI from pre-radiotherapy priors for patients with glioma [0.0]
  Recent developments in artificial intelligence (AI) enable conditional multimodal image generation from clinical data.<n>In this study, we investigate AI-driven generation of follow-up MRI in patients with in- tracranial tumors.
  arXiv  Detail & Related papers  (2026-03-09T13:43:58Z)
• Brain-WM: Brain Glioblastoma World Model [56.45393372490125]
  We present Brain-WM, a pioneering GBM world model that unifies next-step treatment prediction and future MRI generation.<n>Brain-WM encodes latent dynamics into a shared latent space for joint autoregressive treatment prediction and flow-based future MRI generation.<n>Ultimately, Brain-WM offers a robust clinical sandbox for optimizing patient healthcare.
  arXiv  Detail & Related papers  (2026-03-08T09:54:08Z)
• Towards Label-Free Brain Tumor Segmentation: Unsupervised Learning with Multimodal MRI [7.144319861722029]
  Unsupervised anomaly detection (UAD) presents a complementary alternative to supervised learning for brain tumor segmentation in MRI.<n>We propose a novel Multimodal Vision Transformer Autoencoder (MViT-AE) trained exclusively on healthy brain MRIs to detect and localize tumors.<n>Our method achieves clinically meaningful tumor localization, with lesion-wise Dice Similarity Coefficient of 0.437 (Whole Tumor), 0.316 (Tumor Core), and 0.350 (Enhancing Tumor) on the test set, and an anomaly Detection Rate of 89.4% on the validation set.
  arXiv  Detail & Related papers  (2025-10-17T14:26:30Z)
• Glioblastoma Overall Survival Prediction With Vision Transformers [6.318465743962574]
  Glioblastoma is one of the most aggressive and common brain tumors, with a median survival of 10-15 months.<n>In this study, we propose a novel Artificial Intelligence (AI) approach for Overall Survival (OS) prediction using Magnetic Resonance Imaging (MRI) images.<n>We exploit Vision Transformers (ViTs) to extract hidden features directly from MRI images, eliminating the need of tumor segmentation.<n>The proposed model was evaluated on the BRATS dataset, reaching an accuracy of 62.5% on the test set, comparable to the top-performing methods.
  arXiv  Detail & Related papers  (2025-08-04T13:59:57Z)
• GBT-SAM: Adapting a Foundational Deep Learning Model for Generalizable Brain Tumor Segmentation via Efficient Integration of Multi-Parametric MRI Data [5.7802171590699984]
  We present GBT-SAM, a parameter-efficient deep learning framework that adapts the Segment Anything Model to mp-MRI data.<n>Our model is trained by a two-step fine-tuning strategy that incorporates a depth-aware module to capture inter-slice correlations.<n>It achieves a Dice Score of 93.54 on the BraTS Adult Glioma dataset and demonstrates robust performance on Meningioma, Pediatric Glioma, and Sub-Saharan Glioma datasets.
  arXiv  Detail & Related papers  (2025-03-06T11:18:22Z)
• ContextMRI: Enhancing Compressed Sensing MRI through Metadata Conditioning [51.26601171361753]
  We propose ContextMRI, a text-conditioned diffusion model for MRI that integrates granular metadata into the reconstruction process.<n>We show that increasing the fidelity of metadata, ranging from slice location and contrast to patient age, sex, and pathology, systematically boosts reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-08T05:15:43Z)
• Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development [59.74920439478643]
  In this paper, we collect and annotated the first benchmark dataset that covers diverse ERUS scenarios. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. We introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR)
  arXiv  Detail & Related papers  (2024-08-19T15:04:42Z)
• SMRD: SURE-based Robust MRI Reconstruction with Diffusion Models [76.43625653814911]
  Diffusion models have gained popularity for accelerated MRI reconstruction due to their high sample quality. They can effectively serve as rich data priors while incorporating the forward model flexibly at inference time. We introduce SURE-based MRI Reconstruction with Diffusion models (SMRD) to enhance robustness during testing.
  arXiv  Detail & Related papers  (2023-10-03T05:05:35Z)
• ssVERDICT: Self-Supervised VERDICT-MRI for Enhanced Prostate Tumour Characterisation [2.755232740505053]
  Self-supervised neural network for fitting VERDICT estimates parameter maps without training data. We compare the performance of ssVERDICT to two established baseline methods for fitting diffusion MRI models.
  arXiv  Detail & Related papers  (2023-09-12T14:31:33Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• A Long Short-term Memory Based Recurrent Neural Network for Interventional MRI Reconstruction [50.1787181309337]
  We propose a convolutional long short-term memory (Conv-LSTM) based recurrent neural network (RNN), or ConvLR, to reconstruct interventional images with golden-angle radial sampling. The proposed algorithm has the potential to achieve real-time i-MRI for DBS and can be used for general purpose MR-guided intervention.
  arXiv  Detail & Related papers  (2022-03-28T14:03:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.