Vision-Language Model-Guided Deep Unrolling Enables Personalized, Fast MRI

• URL: http://arxiv.org/abs/2604.06849v1
• Date: Wed, 08 Apr 2026 09:10:30 GMT
• Title: Vision-Language Model-Guided Deep Unrolling Enables Personalized, Fast MRI
• Abstract summary: PASS (Personalized, Anomaly-aware Sampling and reconStruction) is an intelligent MRI framework that guides a deep unrolling network for task-oriented, fast imaging.<n> PASS achieves superior image quality across diverse anatomies, contrasts, anomalies, and acceleration factors.<n>This enhancement translates directly to improvements in downstream diagnostic tasks, including fine-grained anomaly detection, localization, and diagnosis.
• Score: 5.617068536645436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Magnetic Resonance Imaging (MRI) is a cornerstone in medicine and healthcare but suffers from long acquisition times. Traditional accelerated MRI methods optimize for generic image quality, lacking adaptability for specific clinical tasks. To address this, we introduce PASS (Personalized, Anomaly-aware Sampling and reconStruction), an intelligent MRI framework that leverages a Vision-Language Model (VLM) to guide a deep unrolling network for task-oriented, fast imaging. PASS dynamically personalizes the imaging pipeline through three core contributions: (1) a deep unrolled reconstruction network derived from a physics-based MRI model; (2) a sampling module that generates patient-specific $k$-space trajectories; and (3) an anomaly-aware prior, extracted from a pretrained VLM, which steers both sampling and reconstruction toward clinically relevant regions. By integrating the high-level clinical reasoning of a VLM with an interpretable, physics-aware network, PASS achieves superior image quality across diverse anatomies, contrasts, anomalies, and acceleration factors. This enhancement directly translates to improvements in downstream diagnostic tasks, including fine-grained anomaly detection, localization, and diagnosis.

Related papers

• Single-Subject Multi-View MRI Super-Resolution via Implicit Neural Representations [4.471119405879635]
  We introduce Single-Subject Implicit Multi-View Super-Resolution for MRI (SIMS-MRI)<n>SIMS-MRI operates solely on anisotropic multi-view scans from a single patient without requiring pre- or post-processing.<n>Our method combines a multi-resolution hash-encoded implicit representation with learned inter-view alignment to generate a spatially consistent isotropic reconstruction.
  arXiv  Detail & Related papers  (2026-03-23T23:00:18Z)
• SSPFormer: Self-Supervised Pretrained Transformer for MRI Images [10.727804908369869]
  This paper proposes a Self-Supervised Pretrained Transformer (SSPFormer) for MRI images.<n>SSPFormer effectively learns domain-specific feature representations of medical images by leveraging unlabeled raw imaging data.<n>Through extensive experiments on segmentation, super-resolution, and denoising tasks, the proposed SSPFormer achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-01-19T06:01:13Z)
• Decipher-MR: A Vision-Language Foundation Model for 3D MRI Representations [12.805804608410739]
  We present Decipher-MR, a 3D MRI-specific vision-language foundation model trained on a large-scale dataset.<n>Decipher-MR integrates self-supervised vision learning with report-guided text supervision to build robust, generalizable representations.<n>Our results establish Decipher-MR as a scalable and versatile foundation for MRI-based AI, facilitating efficient development across clinical and research domains.
  arXiv  Detail & Related papers  (2025-09-25T14:43:33Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervision [24.846428105192405]
  We train M3Ret, a unified visual encoder, without any modality-specific customization.<n>It successfully learns transferable representations using both generative (MAE) and contrastive (SimDINO) self-supervised learning (SSL) paradigms.<n>Our approach sets a new state-of-the-art in zero-shot image-to-image retrieval across all individual modalities, surpassing strong baselines such as DINOv3 and the text-supervised BMC-CLIP.
  arXiv  Detail & Related papers  (2025-09-01T10:59:39Z)
• A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
  We propose a unified MRI reconstruction model robust to various measurement undersampling patterns and image resolutions.<n>Our model improves SSIM by 11% and PSNR by 4 dB over a state-of-the-art CNN (End-to-End VarNet) with 600$times$ faster inference than diffusion methods.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Coordinate-Based Neural Representation Enabling Zero-Shot Learning for 3D Multiparametric Quantitative MRI [4.707353256136099]
  We propose SUMMIT, an innovative imaging methodology that includes data acquisition and an unsupervised reconstruction for simultaneous multiparametric qMRI. The proposed unsupervised approach for qMRI reconstruction also introduces a novel zero-shot learning paradigm for multiparametric imaging applicable to various medical imaging modalities.
  arXiv  Detail & Related papers  (2024-10-02T14:13:06Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
  We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
  arXiv  Detail & Related papers  (2022-09-19T09:43:19Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.