MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI

• URL: http://arxiv.org/abs/2608.30975v1
• Date: Mon, 31 Aug 2026 15:36:24 GMT
• Title: MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI
• Abstract summary: We present MR-JEPA, a self-supervised video foundation model for CMR.<n>Unlike prior CMR video models limited to cine data, MR-JEPA is pretrained on sequential multi-sequence data.
• Score: 21.245485191371703
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Cardiac magnetic resonance imaging (CMR) produces rich sequential data such as temporal cine videos and spatial LGE/mapping stacks, yet most deep learning approaches process individual 2D slices, discarding this context. We present MR-JEPA, a self-supervised video foundation model for CMR that extends LeJEPA to 3D spatiotemporal inputs through tubelet tokenization, spatiotemporal masking augmentation, and initialization from a 2D CMR foundation model. Unlike prior CMR video models limited to cine data, MR-JEPA is pretrained on multi-sequence data (cine, LGE, mapping) from 10,505 patients across two centers without annotations. We evaluate the frozen encoder on six downstream tasks using a unified multi-view gated attention architecture: LV ejection fraction, RV ejection fraction, three myocardial strains (GLS, GCS, GRS), and four-class disease detection. MR-JEPA outperforms other compared methods on all five regression tasks, including both a domain-specific CMR model pretrained on more data with text supervision and a natural-video foundation model, achieving an LV EF MAE of 4.79% (r =0.764) and a GLS MAE of 1.87 (r=0.805), with 21-27% MAE reductions over baselines on strain tasks. For disease detection, MR-JEPA achieved a macro AUG of 0.868, remaining competitive with the domain-specific baseline despite using a fully self-supervised pretraining objective. These results demonstrate the potential of a unified video encoder for robust, multi-view utilization of diverse CMR sequences in clinical cardiac quantification and diagnosis.

Related papers

• CMRVision: A Foundation Model for Cardiac MR Image Analysis [21.180056139848528]
  We introduce CMRVision, a CMR-specific foundation model trained using DINOv3-style self-supervised learning.<n>CMRVision is evaluated on two downstream tasks: multi-task segmentation across cine, late gadolinium enhancement (LGE) and mapping sequences.<n>Our experiments show that CMR-specific pretraining, smaller patch sizes, and patch-level objectives consistently improve downstream performance.
  arXiv  Detail & Related papers  (2026-09-01T14:34:30Z)
• Foundation Models Adaptation for Multi-View Multi-modal Cardiac MRI Segmentation and Direct Ejection Fraction Estimation [0.3957682607484472]
  Foundation models have shown strong transferability in cardiac MRI (CMR), but their effectiveness for heterogeneous multi-view and multi-sequence analysis remains unclear.<n>We explore the effectiveness of fine-tuning and combining different CMR foundation models for the Universal Multi-Sequence, Multi-Center and Multi-View CMR (CMR-Multi) Challenge.
  arXiv  Detail & Related papers  (2026-08-07T14:48:51Z)
• A physics-informed foundation model for quantitative diffusion MRI [46.74037857287099]
  A physics-informed generative microstructure network (PIGMENT) learns a universal generative prior to human brain microstructure and adapts it zero-shot to each participant's measured data to recover subject-specific maps.<n>PIGMENT estimates demonstrated strong biological validity, preserving submillimeter cortical microarchitectural patterns and early-childhood white matter developmental trajectories from 10-fold accelerated scans.<n>These results establish PIGMENT as a physics-informed foundation model that extends quantitative diffusion MRI into regimes traditionally too sparse, heterogeneous, or clinically constrained for reliable analysis.
  arXiv  Detail & Related papers  (2026-05-29T07:26:35Z)
• Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation [0.0]
  Multi-weighted prostate MRI combines T2-parametric (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences.<n>In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W.<n>We propose Modality-Isolated Gated Fusion (MIGF) to maintain separate modality-specific encoding streams before a learned gating stage.
  arXiv  Detail & Related papers  (2026-04-12T15:54:21Z)
• Scaling Video Pretraining for Surgical Foundation Models [51.92777479821822]
  SurgRec is a scalable and reproducible pretraining recipe for surgical video understanding.<n>We curate a large corpus of 10,535 videos and 214.5M frames spanning endoscopy, laparoscopy, cataract, and robotic surgery.
  arXiv  Detail & Related papers  (2026-03-31T16:31:25Z)
• PRISM: Performer RS-IMLE for Single-pass Multisensory Imitation Learning [51.24484551729328]
  We introduce PRISM, a single-pass policy based on a batch-global rejection-sampling variant of IMLE.<n> PRISM couples a temporal multisensory encoder with a linear-attention generator using a Performer architecture.<n>We demonstrate the efficacy of PRISM on a diverse real-world hardware suite, including loco-manipulation using a Unitree Go2 with a 7-DoF arm D1 and tabletop manipulation with a UR5 manipulator.
  arXiv  Detail & Related papers  (2026-02-02T17:57:37Z)
• Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency [60.74505433956616]
  continuous-time consistency model (sCM) is theoretically principled and empirically powerful for accelerating academic-scale diffusion.<n>We first develop a parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on models with over 10 billion parameters and high-dimensional video tasks.<n>We propose the score-regularized continuous-time consistency model (rCM), which incorporates score distillation as a long-skip regularizer.
  arXiv  Detail & Related papers  (2025-10-09T16:45:30Z)
• LGE-Guided Cross-Modality Contrastive Learning for Gadolinium-Free Cardiomyopathy Screening in Cine CMR [51.11296719862485]
  We propose a Contrastive Learning and Cross-Modal alignment framework for gadolinium-free cardiomyopathy screening using cine CMR sequences.<n>By aligning the latent spaces of cine CMR and Late Gadolinium Enhancement (LGE) sequences, our model encodes fibrosis-specific pathology into cine CMR embeddings.
  arXiv  Detail & Related papers  (2025-08-23T07:21:23Z)
• Improving the U-Net Configuration for Automated Delineation of Head and Neck Cancer on MRI [0.0]
  Tumor volume segmentation on MRI is a challenging and time-consuming process.<n>This work presents an approach to automated delineation of head and neck tumors on MRI scans.<n>The focus of this research was to propose improvements to the configuration commonly used in medical segmentation tasks.
  arXiv  Detail & Related papers  (2025-01-09T10:22:35Z)
• Deep Learning for Longitudinal Gross Tumor Volume Segmentation in MRI-Guided Adaptive Radiotherapy for Head and Neck Cancer [4.358109501717511]
  Accurate segmentation of gross tumor volume (GTV) is essential for effective MRI-guided adaptive radiotherapy (MRgART) in head and neck cancer.<n>In this study, we tackled the challenges of both pre-radiotherapy (pre-RT) and mid-radiotherapy (mid-RT) tumor volume segmentation.<n>We presented a collection of DL models that could facilitate GTV segmentation in MRgART, offering the potential to streamline radiation oncology.
  arXiv  Detail & Related papers  (2024-12-01T03:57:18Z)
• Lung Cancer Lesion Detection in Histopathology Images Using Graph-Based Sparse PCA Network [93.22587316229954]
  We propose a graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E) We evaluate the performance of the proposed algorithm on H&E slides obtained from an SVM K-rasG12D lung cancer mouse model using precision/recall rates, F-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC)
  arXiv  Detail & Related papers  (2021-10-27T19:28:36Z)
• Deep Learning Estimation of Multi-Tissue Constrained Spherical Deconvolution with Limited Single Shell DW-MRI [2.903217519429591]
  Deep learning can be used to estimate the information content captured by 8th order constrained spherical deconvolution (CSD) We examine two network architectures: Sequential network of fully connected dense layers with a residual block in the middle (ResDNN), and Patch based convolutional neural network with a residual block (ResCNN) The fiber orientation distribution function (fODF) can be recovered with high correlation as compared to the ground truth of MT-CST, which was derived from the multi-shell DW-MRI acquisitions.
  arXiv  Detail & Related papers  (2020-02-20T15:59:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.