Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy

• URL: http://arxiv.org/abs/2607.03959v1
• Date: Sat, 04 Jul 2026 17:26:30 GMT
• Title: Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy
• Abstract summary: Diabetic retinopathy is a leading cause of vision impairment worldwide.<n>We propose a cross-modal fusion of OCT B-scans with single-channel en face OCTA using a bidirectional cross-modal attention network for automated DR classification.
• Score: 0.2392302093341198
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Diabetic retinopathy (DR) is a leading cause of vision impairment worldwide, highlighting the need for accurate and accessible screening tools. Optical Coherence Tomography (OCT) provides high-resolution structural information of the retina, whereas OCT angiography (OCTA) offers complementary vascular information that is highly relevant for DR diagnosis. In this study, we propose a cross-modal fusion of OCT B-scans with single-channel en face OCTA using a bidirectional cross-modal attention network for automated DR classification. Two independent datasets, OCT500 and UIC, comprising 730 subjects in total, were utilized to evaluate performance under within-dataset, combined-dataset, and cross-dataset generalization settings. A ConvNeXt V2 model trained solely on OCT images served as the unimodal baseline. In addition to ground-truth (GT) OCTA, we explored the use of translated (TR) OCTA generated from OCT scans, eliminating the requirement for dedicated OCTA hardware. Experimental results demonstrate that cross-modal fusion consistently outperforms unimodal OCT classification across all evaluation scenarios. Fusion with GT OCTA improved classification accuracy and discriminative performance, while TR OCTA achieved comparable or superior results in most settings. Furthermore, TR OCTA improved sensitivity and cross-dataset generalization, indicating enhanced robustness to domain shifts. These findings demonstrate that attention-based OCT-OCTA en face fusion provides clinically meaningful improvements for DR detection and suggest that computationally generated OCTA can serve as a practical, low-cost alternative to hardware-acquired OCTA, enabling broader deployment of high-performance retinal screening systems in resource-limited clinical environments.

Related papers

• Automated 2D and 3D Segmentation of AMD and DME Lesions in OCT [5.625893116312849]
  Age-related macular degeneration and diabetic macular edema are leading causes of vision loss.<n>Most deep-learning segmentation work for optical coherence tomography is validated only in-domain.<n>This work develops and systematically ablates four lesion-segmentation pipelines.
  arXiv  Detail & Related papers  (2026-08-27T13:13:18Z)
• Distilling Photon-Counting CT into Routine Chest CT through Clinically Validated Degradation Modeling [51.330452704464925]
  We train a latent diffusion model on 1,046 PCCTs and 405,379 EICTs from 145 hospitals.<n>We construct a large-scale dataset of over 17,316 publicly available EICTs enhanced to PCCT-like quality.<n>Our results suggest emerging imaging advances can be systematically distilled into routine EICT using limited high-quality scans as reference.
  arXiv  Detail & Related papers  (2026-04-08T17:47:35Z)
• PHASOR: Anatomy- and Phase-Consistent Volumetric Diffusion for CT Virtual Contrast Enhancement [60.81732730684265]
  This paper introduces PHASOR, a volumetric diffusion framework for high-fidelity virtual contrast enhancement (VCE)<n>By treating CT volumes as coherent sequences, we leverage a video diffusion model to enhance structural coherence and volumetric accuracy.<n>To ensure anatomy-phase consistent synthesis, we introduce two complementary modules. First, anatomy-routed mixture-of-experts (AR-MoE) anchors distinct enhancement patterns to anatomical semantics, with organ-specific memory to capture salient details.<n>Second, intensity-phase aware representation alignment (IP-REPA) highlights intricate contrast signals while mitigating the impact of imperfect spatial alignment.
  arXiv  Detail & Related papers  (2026-04-01T15:57:18Z)
• KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification [0.0]
  Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide.<n> deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands.<n>It is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment.
  arXiv  Detail & Related papers  (2025-12-09T19:34:30Z)
• EqDiff-CT: Equivariant Conditional Diffusion model for CT Image Synthesis from CBCT [43.92108185590778]
  Cone-beam computed tomography (CBCT) is widely used for imageguided radiotherapy (IGRT)<n>We propose a novel diffusion-based conditional generative model, coined EqDiff-CT, to synthesize high-quality CT images from CBCT.
  arXiv  Detail & Related papers  (2025-09-26T05:51:59Z)
• XOCT: Enhancing OCT to OCTA Translation via Cross-Dimensional Supervised Multi-Scale Feature Learning [10.638764687563555]
  X OCT is a novel deep learning framework that integrates Cross-Dimensional Supervision (CDS) with a Multi-Scale Feature Fusion (MSFF) network for vascular reconstruction.<n>Our experiments on the OCTA-500 dataset demonstrate X OCT's improvements, especially for the en-face projections.
  arXiv  Detail & Related papers  (2025-09-09T07:25:36Z)
• Adapting Foundation Model for Dental Caries Detection with Dual-View Co-Training [53.77904429789069]
  We present Attention-TNet, a novel Dual-View Co-Training network for accurate dental caries detection.<n>OurTNet starts with employing automated tooth detection to establish two complementary views: a global view from panoramic X-ray images and a local view from cropped tooth images.<n>To effectively integrate information from both views, we introduce a Gated Cross-View module.
  arXiv  Detail & Related papers  (2025-08-28T14:13:26Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• Comprehensive Evaluation of OCT-based Automated Segmentation of Retinal Layer, Fluid and Hyper-Reflective Foci: Impact on Clinical Assessment of Diabetic Retinopathy Severity [0.0]
  Diabetic retinopathy (DR) is a leading cause of vision loss, requiring early and accurate assessment to prevent irreversible damage.<n>This study proposes an active-learning-based deep learning pipeline for automated segmentation of retinal layers.
  arXiv  Detail & Related papers  (2025-03-03T07:23:56Z)
• DPER: Diffusion Prior Driven Neural Representation for Limited Angle and Sparse View CT Reconstruction [45.00528216648563]
  Diffusion Prior Driven Neural Representation (DPER) is an unsupervised framework designed to address the exceptionally ill-posed CT reconstruction inverse problems. DPER adopts the Half Quadratic Splitting (HQS) algorithm to decompose the inverse problem into data fidelity and distribution prior sub-problems. We conduct comprehensive experiments to evaluate the performance of DPER on LACT and ultra-SVCT reconstruction with two public datasets.
  arXiv  Detail & Related papers  (2024-04-27T12:55:13Z)
• Quantitative Characterization of Retinal Features in Translated OCTA [0.6664270117164767]
  This study explores the feasibility of using generative machine learning (ML) to translate Optical Coherence Tomography ( OCT) images into Optical Coherence Tomography Angiography ( OCTA) images.
  arXiv  Detail & Related papers  (2024-04-24T18:40:45Z)
• Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation [48.638327652506284]
  Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. We present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach.
  arXiv  Detail & Related papers  (2022-11-12T05:39:17Z)
• Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis [88.39466012709205]
  This paper builds a novel medical slice to increase the between-slice resolution. Considering that the ground-truth intermediate medical slices are always absent in clinical practice, we introduce the incremental cross-view mutual distillation strategy. Our method outperforms state-of-the-art algorithms by clear margins.
  arXiv  Detail & Related papers  (2021-12-20T03:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.