OphMAE: Bridging Volumetric and Planar Imaging with a Foundation Model for Adaptive Ophthalmological Diagnosis

• URL: http://arxiv.org/abs/2605.02714v1
• Date: Mon, 04 May 2026 15:18:06 GMT
• Title: OphMAE: Bridging Volumetric and Planar Imaging with a Foundation Model for Adaptive Ophthalmological Diagnosis
• Abstract summary: Ophthalmic multimodal Masked Autoencoder (OphMAE) is a multi-imaging foundation model engineered to synergize the volumetric depth of 3D Optical Coherence Tomography ( OCT) with the planar context of 2D en face OCT.<n>The model demonstrated state-of-the-art performance, achieving an Area Under the Curve (AUC) of 96.9% for Age-related Macular Degeneration (AMD) and 97.2% for Diabetic Macular Edema (DME)<n>This work establishes a scalable and adaptable framework for ophthalmic AI, ensuring robust performance across different tasks.
• Score: 13.730900699529698
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of foundation models has heralded a new era in medical artificial intelligence (AI), enabling the extraction of generalizable representations from large-scale unlabeled datasets. However, current ophthalmic AI paradigms are predominantly constrained to single-modality inference, thereby creating a dissonance with clinical practice where diagnosis relies on the synthesis of complementary imaging modalities. Furthermore, the deployment of high-performance AI in resource-limited settings is frequently impeded by the unavailability of advanced three-dimensional imaging hardware. Here, we present the Ophthalmic multimodal Masked Autoencoder (OphMAE), a multi-imaging foundation model engineered to synergize the volumetric depth of 3D Optical Coherence Tomography (OCT) with the planar context of 2D en face OCT. By implementing a novel cross-modal fusion architecture and a unique adaptive inference mechanism, OphMAE was pre-trained on a massive dataset with of 183,875 paired OCT images derived from 32,765 patients. In a rigorous benchmark encompassing 17 diverse diagnostic tasks with 48,340 paired OCT images from 8,191 patients, the model demonstrated state-of-the-art performance, achieving an Area Under the Curve (AUC) of 96.9% for Age-related Macular Degeneration (AMD) and 97.2% for Diabetic Macular Edema (DME), consistently surpassing existing single-modal and multimodal foundation models. Crucially, OphMAE exhibits robust engineering adaptability: it maintains high diagnostic accuracy, such as 93.7\% AUC for AMD, even when restricted to single-modality 2D inputs, and demonstrates exceptional data efficiency by retaining 95.7% AUC with as few as 500 labeled samples. This work establishes a scalable and adaptable framework for ophthalmic AI, ensuring robust performance across different tasks.

Related papers

• Prob-BBDM: a Probabilistic Brownian Bridge Diffusion Model for MRI sequence image-to-image translation [1.8036000511825068]
  We propose a novel image-to-image translation model based on Brownian Bridge Diffusion Models (BBDM)<n>Our approach integrates a variational encoder-guided diffusion mechanism, leveraging probabilistic image distributions to enhance synthesis quality.<n>Prob-BBDM achieves superior performance across multiple translation tasks, reaching up to 88.46% SSIM and 26.09 dB PSNR.
  arXiv  Detail & Related papers  (2026-06-23T08:47:17Z)
• D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities [54.87947751720332]
  We propose a novel segmentation model D3Seg to maintain stable performance under missing-modality settings.<n>The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing modality configurations.
  arXiv  Detail & Related papers  (2026-05-21T09:55:42Z)
• LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol [9.075009508165273]
  This dataset contains 1824 images from 468 patients with pathology-confirmed labels, BI-RADS assessments, and breast-density annotations.<n>LUMINA spans six acquisition systems and includes both high- and low-energy imaging styles.<n>We benchmark CNN and transformer models on three clinically relevant tasks: diagnosis, BI-RADS classification, and density estimation.
  arXiv  Detail & Related papers  (2026-03-15T22:41:40Z)
• UniField: A Unified Field-Aware MRI Enhancement Framework [45.03230466428695]
  methodname is a unified framework integrating multiple modalities and enhancement tasks.<n>We exploit comprehensive 3D volumetric information by leveraging pre-trained 3D foundation models.<n>We organize and publicly release a comprehensive paired multi-field MRI dataset.
  arXiv  Detail & Related papers  (2026-03-10T05:45:12Z)
• A Contrastive Variational AutoEncoder for NSCLC Survival Prediction with Missing Modalities [41.8469011437549]
  Predicting survival outcomes for non-small cell lung cancer (NSCLC) patients is challenging due to the different individual prognostic features.<n>State-of-the-art models rely on available data to create patient-level representations or use generative models to infer missing modalities.<n>We propose a Multimodal Contrastive Variational AutoEncoder (MCVAE) to address this issue.
  arXiv  Detail & Related papers  (2026-02-19T14:29:34Z)
• A multimodal vision foundation model for generalizable knee pathology [40.03838145472935]
  Musculoskeletal disorders represent an urgent demand for precise interpretation of medical imaging.<n>Current artificial intelligence approaches in orthopedics rely on task-specific, supervised learning paradigms.<n>We introduce OrthoFoundation, a multimodal vision foundation model optimized for musculoskeletal pathology.
  arXiv  Detail & Related papers  (2026-01-26T08:14:51Z)
• Toward explainable AI approaches for breast imaging: adapting foundation models to diverse populations [4.505150709006532]
  Foundation models hold promise for specialized medical imaging tasks, though their effectiveness in breast imaging remains underexplored.<n>This study leverages BiomedCLIP as a foundation model to address challenges in model generalization.<n>Using 96,995 images, we compared single-modality (s2D only) and multi-modality training approaches, addressing class imbalance through weighted contrastive learning.
  arXiv  Detail & Related papers  (2025-11-21T22:45:50Z)
• Improved mmFormer for Liver Fibrosis Staging via Missing-Modality Compensation [8.687370165870613]
  We propose a multimodal MRI classification model based on the mmFormer architecture with an adaptive module for handling arbitrary combinations of missing modalities.<n>This method is evaluated on the test set of Comprehensive Analysis & Computing of REal-world medical images (CARE 2025 challenge)<n>For Cirrhosis Detection and Substantial Fibrosis Detection on in-distribution vendors, our model obtains accuracies of 66.67%, and 74.17%, and corresponding area under the curve (AUC) scores of 71.73% and 68.48%, respectively.
  arXiv  Detail & Related papers  (2025-09-19T21:31:05Z)
• APTOS-2024 challenge report: Generation of synthetic 3D OCT images from fundus photographs [42.58128666405841]
  The Asia Pacific Tele-Ophthalmology Society organized a challenge titled Artificial Intelligence-based OCT Generation from Fundus Images.<n>This paper details the challenge framework (referred to as APTOS-2024 Challenge), including the benchmark dataset.<n>The challenge attracted 342 participating teams, with 42 preliminary submissions and 9 finalists.
  arXiv  Detail & Related papers  (2025-06-09T08:29:37Z)
• Latent Diffusion Autoencoders: Toward Efficient and Meaningful Unsupervised Representation Learning in Medical Imaging [41.446379453352534]
  Latent Diffusion Autoencoder (LDAE) is a novel encoder-decoder diffusion-based framework for efficient and meaningful unsupervised learning in medical imaging.<n>This study focuses on Alzheimer disease (AD) using brain MR from the ADNI database as a case study.
  arXiv  Detail & Related papers  (2025-04-11T15:37:46Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis [1.647759094903376]
  This work presents WDM, a wavelet-based medical image synthesis framework that applies a diffusion model on wavelet images. Experimental results on BraTS and LIDC-IDRI unconditional image generation at a resolution of $128 times 128 times 128$ demonstrate state-of-the-art image fidelity (FID) and sample diversity (MS-SSIM) scores. Our proposed method is the only one capable of generating high-quality images at a resolution of $256 times 256 times 256$, outperforming all comparing methods.
  arXiv  Detail & Related papers  (2024-02-29T11:11:05Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.