Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management

• URL: http://arxiv.org/abs/2502.13182v1
• Date: Tue, 18 Feb 2025 10:02:39 GMT
• Title: Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management
• Authors: Danli Shi, Bowen Liu, Zhen Tian, Yue Wu, Jiancheng Yang, Ruoyu Chen, Bo Yang, Ou Xiao, Mingguang He,
• Abstract summary: Myopia, projected to affect 50% population globally by 2050, is a leading cause of vision loss. Recent understanding of eye-shape-based biomarkers requires magnetic resonance imaging (MRI) Fundus2Globe is the first AI framework that synthesizes patient-specific 3D eye globes from 2D color fundus photographs.
• Score: 17.197828632845198
• License:
• Abstract: Myopia, projected to affect 50% population globally by 2050, is a leading cause of vision loss. Eyes with pathological myopia exhibit distinctive shape distributions, which are closely linked to the progression of vision-threatening complications. Recent understanding of eye-shape-based biomarkers requires magnetic resonance imaging (MRI), however, it is costly and unrealistic in routine ophthalmology clinics. We present Fundus2Globe, the first AI framework that synthesizes patient-specific 3D eye globes from ubiquitous 2D color fundus photographs (CFPs) and routine metadata (axial length, spherical equivalent), bypassing MRI dependency. By integrating a 3D morphable eye model (encoding biomechanical shape priors) with a latent diffusion model, our approach achieves submillimeter accuracy in reconstructing posterior ocular anatomy efficiently. Fundus2Globe uniquely quantifies how vision-threatening lesions (e.g., staphylomas) in CFPs correlate with MRI-validated 3D shape abnormalities, enabling clinicians to simulate posterior segment changes in response to refractive shifts. External validation demonstrates its robust generation performance, ensuring fairness across underrepresented groups. By transforming 2D fundus imaging into 3D digital replicas of ocular structures, Fundus2Globe is a gateway for precision ophthalmology, laying the foundation for AI-driven, personalized myopia management.

Related papers

• Is an Ultra Large Natural Image-Based Foundation Model Superior to a Retina-Specific Model for Detecting Ocular and Systemic Diseases? [15.146396276161937]
  RETFound and DINOv2 models were evaluated for ocular disease detection and systemic disease prediction tasks. RETFound achieved superior performance over all DINOv2 models in predicting heart failure, infarction, and ischaemic stroke.
  arXiv  Detail & Related papers  (2025-02-10T09:31:39Z)
• Spatial-aware Transformer-GRU Framework for Enhanced Glaucoma Diagnosis from 3D OCT Imaging [1.8416014644193066]
  We present a novel deep learning framework that leverages the diagnostic value of 3D Optical Coherence Tomography ( OCT) imaging for automated glaucoma detection. We integrate a pre-trained Vision Transformer on retinal data for rich slice-wise feature extraction and a bidirectional Gated Recurrent Unit for capturing inter-slice spatial dependencies. Experimental results on a large dataset demonstrate the superior performance of the proposed method over state-of-the-art ones.
  arXiv  Detail & Related papers  (2024-03-08T22:25:15Z)
• Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples [6.381153836752796]
  We present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA) for processing 3D tissue images. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias.
  arXiv  Detail & Related papers  (2023-07-27T14:48:02Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• Deep learning network to correct axial and coronal eye motion in 3D OCT retinal imaging [65.47834983591957]
  We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single scan. The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods.
  arXiv  Detail & Related papers  (2023-05-27T03:55:19Z)
• nnUNet RASPP for Retinal OCT Fluid Detection, Segmentation and Generalisation over Variations of Data Sources [25.095695898777656]
  We propose two variants of the nnUNet with consistent high performance across images from multiple device vendors. The algorithm was validated on the MICCAI 2017 RETOUCH challenge dataset. Experimental results show that our algorithms outperform the current state-of-the-arts algorithms.
  arXiv  Detail & Related papers  (2023-02-25T23:47:23Z)
• View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
  We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection. First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan. Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view. Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
  arXiv  Detail & Related papers  (2022-09-20T11:58:23Z)
• Geometric Deep Learning to Identify the Critical 3D Structural Features of the Optic Nerve Head for Glaucoma Diagnosis [52.06403518904579]
  The optic nerve head (ONH) undergoes complex and deep 3D morphological changes during the development and progression of glaucoma. We used PointNet and dynamic graph convolutional neural network (DGCNN) to diagnose glaucoma from 3D ONH point clouds. Our approach may have strong potential to be used in clinical applications for the diagnosis and prognosis of a wide range of ophthalmic disorders.
  arXiv  Detail & Related papers  (2022-04-14T12:52:10Z)
• Multi-Scale Convolutional Neural Network for Automated AMD Classification using Retinal OCT Images [1.299941371793082]
  Age-related macular degeneration (AMD) is the most common cause of blindness in developed countries, especially in people over 60 years of age. Recent developments in deep learning have provided a unique opportunity for the development of fully automated diagnosis frameworks. We propose a multi-scale convolutional neural network (CNN) capable of distinguishing pathologies using receptive fields with various sizes.
  arXiv  Detail & Related papers  (2021-10-06T18:20:58Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.