Freqformer: Frequency-Domain Transformer for 3-D Reconstruction and Quantification of Human Retinal Vasculature

• URL: http://arxiv.org/abs/2411.11189v2
• Date: Fri, 26 Sep 2025 21:21:35 GMT
• Title: Freqformer: Frequency-Domain Transformer for 3-D Reconstruction and Quantification of Human Retinal Vasculature
• Authors: Lingyun Wang, Bingjie Wang, Jay Chhablani, Jose Alain Sahel, Shaohua Pi,
• Abstract summary: We introduce Freqformer, a novel Transformer-based model featuring a dual-branch architecture that integrates a Transformer layer for capturing global spatial context.<n>Freqformer was trained using single depth-plane OCTA images, utilizing volumetrically merged OCTA as the ground truth.<n>Freqformer substantially outperformed existing convolutional neural networks and Transformer-based methods, achieving superior image metrics.
• Score: 3.708884194494243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Objective: To achieve accurate 3-D reconstruction and quantitative analysis of human retinal vasculature from a single optical coherence tomography angiography (OCTA) scan. Methods: We introduce Freqformer, a novel Transformer-based model featuring a dual-branch architecture that integrates a Transformer layer for capturing global spatial context with a complex-valued frequency-domain module designed for adaptive frequency enhancement. Freqformer was trained using single depth-plane OCTA images, utilizing volumetrically merged OCTA as the ground truth. Performance was evaluated quantitatively through 2-D and 3-D image quality metrics. 2-D networks and their 3-D counterparts were compared to assess the differences between enhancing volume slice by slice and enhancing it by 3-D patches. Furthermore, 3-D quantitative vascular metrics were conducted to quantify human retinal vasculature. Results: Freqformer substantially outperformed existing convolutional neural networks and Transformer-based methods, achieving superior image metrics. Importantly, the enhanced OCTA volumes show strong correlation with the merged volumes on vascular segment count, density, length, and flow index, further underscoring its reliability for quantitative vascular analysis. 3-D counterparts did not yield additional gains in image metrics or downstream 3-D vascular quantification but incurred nearly an order-of-magnitude longer inference time, supporting our 2-D slice-wise enhancement strategy. Additionally, Freqformer showed excellent generalization capability on larger field-of-view scans, surpassing the quality of conventional volumetric merging methods. Conclusion: Freqformer reliably generates high-definition 3-D retinal microvasculature from single-scan OCTA, enabling precise vascular quantification comparable to standard volumetric merging methods.

Related papers

• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• Implicit Spatiotemporal Bandwidth Enhancement Filter by Sine-activated Deep Learning Model for Fast 3D Photoacoustic Tomography [0.0]
  3D photoacoustic tomography (3D-PAT) using high-frequency hemispherical transducers offers near-omnidirectional reception.<n>However, practical constraints such as limited number of channels with bandlimited sampling rate often result in sparse and bandlimited sensors that degrade image quality.<n>We revisit the 2D deep learning (DL) approach applied directly to sensor-wise PA radio-frequency (PARF) data.<n>Specifically, we introduce sine-activated into the DL model to restore the broadband nature of PARF signals.
  arXiv  Detail & Related papers  (2025-07-28T07:16:32Z)
• High Volume Rate 3D Ultrasound Reconstruction with Diffusion Models [14.007660129016852]
  Three-dimensional ultrasound enables real-time volumetric visualization of anatomical structures.<n> achieving both high volume rates and high image quality remains a significant challenge.<n>This paper introduces a novel approach to 3D ultrasound reconstruction by employing diffusion models (DMs) to achieve increased spatial and temporal resolution.
  arXiv  Detail & Related papers  (2025-05-28T08:14:12Z)
• DeepSPV: A Deep Learning Pipeline for 3D Spleen Volume Estimation from 2D Ultrasound Images [7.727342487565146]
  Splenomegaly is an important clinical indicator for various associated medical conditions, such as sickle cell disease (SCD)<n>In this work, we introduce a deep learning pipeline, DeepSPV, for precise spleen volume estimation from single or dual 2D ultrasound images.
  arXiv  Detail & Related papers  (2024-11-17T22:43:07Z)
• Deep-Motion-Net: GNN-based volumetric organ shape reconstruction from single-view 2D projections [1.8189671456038365]
  We propose an end-to-end graph neural network architecture that enables 3D organ shape reconstruction during radiotherapy. The proposed model learns the mesh regression from a patient-specific template and deep features extracted from kV images at arbitrary projection angles. Overall framework was tested quantitatively on synthetic respiratory motion scenarios and qualitatively on in-treatment images acquired over full scan series for liver cancer patients.
  arXiv  Detail & Related papers  (2024-07-09T09:07:18Z)
• Diff3Dformer: Leveraging Slice Sequence Diffusion for Enhanced 3D CT Classification with Transformer Networks [5.806035963947936]
  We propose a Diffusion-based 3D Vision Transformer (Diff3Dformer) to aggregate repetitive information within 3D CT scans. Our method exhibits improved performance on two different scales of small datasets of 3D lung CT scans.
  arXiv  Detail & Related papers  (2024-06-24T23:23:18Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D [88.66678730537777]
  We present StableDreamer, a methodology incorporating three advances. First, we formalize the equivalence of the SDS generative prior and a simple supervised L2 reconstruction loss. Second, our analysis shows that while image-space diffusion contributes to geometric precision, latent-space diffusion is crucial for vivid color rendition.
  arXiv  Detail & Related papers  (2023-12-02T02:27:58Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows [16.928404625892625]
  Real-time 3D fluorescence microscopy is crucial for the analysis of live organisms. We propose a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity.
  arXiv  Detail & Related papers  (2023-06-10T10:42:49Z)
• Image Reconstruction for Accelerated MR Scan with Faster Fourier Convolutional Neural Networks [87.87578529398019]
  Partial scan is a common approach to accelerate Magnetic Resonance Imaging (MRI) data acquisition in both 2D and 3D settings. We propose a novel convolutional operator called Faster Fourier Convolution (FasterFC) to replace the two consecutive convolution operations. A 2D accelerated MRI method, FasterFC-End-to-End-VarNet, which uses FasterFC to improve the sensitivity maps and reconstruction quality. A 3D accelerated MRI method called FasterFC-based Single-to-group Network (FAS-Net) that utilizes a single-to-group algorithm to guide k-space domain reconstruction
  arXiv  Detail & Related papers  (2023-06-05T13:53:57Z)
• View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
  We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection. First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan. Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view. Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
  arXiv  Detail & Related papers  (2022-09-20T11:58:23Z)
• Deep learning of multi-resolution X-Ray micro-CT images for multi-scale modelling [0.0]
  We develop a 3D Enhanced Deep Super Resolution (EDSR) convolutional neural network to create enhanced, high-resolution data over large spatial scales. We validate the network with various metrics: textual analysis, segmentation behaviour and pore-network model (PNM) multiphase flow simulations. The EDSR generated model is more accurate than the base LR model at predicting experimental behaviour in the presence of heterogeneities.
  arXiv  Detail & Related papers  (2021-11-01T21:49:22Z)
• End-to-end Ultrasound Frame to Volume Registration [9.738024231762465]
  We propose an end-to-end frame-to-volume registration network (FVR-Net) for 2D and 3D registration. Our model shows superior efficiency for real-time interventional guidance with highly competitive registration accuracy.
  arXiv  Detail & Related papers  (2021-07-14T01:59:42Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE [66.63629641650572]
  We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices. We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
  arXiv  Detail & Related papers  (2020-07-09T13:23:15Z)
• 3D Solid Spherical Bispectrum CNNs for Biomedical Texture Analysis [3.579867431007686]
  Locally Rotation Invariant (LRI) operators have shown great potential in biomedical texture analysis. We investigate the benefits of using the bispectrum over the spectrum in the design of a LRI layer embedded in a shallow Convolutional Neural Network (CNN) for 3D image analysis.
  arXiv  Detail & Related papers  (2020-04-28T09:01:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.