Freqformer: Frequency-Domain Transformer for 3-D Visualization and Quantification of Human Retinal Circulation

• URL: http://arxiv.org/abs/2411.11189v1
• Date: Sun, 17 Nov 2024 22:38:39 GMT
• Title: Freqformer: Frequency-Domain Transformer for 3-D Visualization and Quantification of Human Retinal Circulation
• Authors: Lingyun Wang, Bingjie Wang, Jay Chhablani, Jose Alain Sahel, Shaohua Pi,
• Abstract summary: Freqformer is a Transformer-based architecture designed for 3-D, high-definition visualization of human retinal circulation from a single scan. Our method outperforms state-of-the-art convolutional neural networks (CNNs) and several Transformer-based models. Freqformer can significantly improve the understanding and characterization of retinal circulation, offering potential clinical applications.
• Score: 0.9487097819140653
• License:
• Abstract: We introduce Freqformer, a novel Transformer-based architecture designed for 3-D, high-definition visualization of human retinal circulation from a single scan in commercial optical coherence tomography angiography (OCTA). Freqformer addresses the challenge of limited signal-to-noise ratio in OCTA volume by utilizing a complex-valued frequency-domain module (CFDM) and a simplified multi-head attention (Sim-MHA) mechanism. Using merged volumes as ground truth, Freqformer enables accurate reconstruction of retinal vasculature across the depth planes, allowing for 3-D quantification of capillary segments (count, density, and length). Our method outperforms state-of-the-art convolutional neural networks (CNNs) and several Transformer-based models, with superior performance in peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). Furthermore, Freqformer demonstrates excellent generalizability across lower scanning density, effectively enhancing OCTA scans with larger fields of view (from 3$\times$3 $mm^{2}$ to 6$\times$6 $mm^{2}$ and 12$\times$12 $mm^{2}$). These results suggest that Freqformer can significantly improve the understanding and characterization of retinal circulation, offering potential clinical applications in diagnosing and managing retinal vascular diseases.

Related papers

• Deep-Motion-Net: GNN-based volumetric organ shape reconstruction from single-view 2D projections [1.8189671456038365]
  We propose an end-to-end graph neural network architecture that enables 3D organ shape reconstruction during radiotherapy. The proposed model learns the mesh regression from a patient-specific template and deep features extracted from kV images at arbitrary projection angles. Overall framework was tested quantitatively on synthetic respiratory motion scenarios and qualitatively on in-treatment images acquired over full scan series for liver cancer patients.
  arXiv  Detail & Related papers  (2024-07-09T09:07:18Z)
• Diff3Dformer: Leveraging Slice Sequence Diffusion for Enhanced 3D CT Classification with Transformer Networks [5.806035963947936]
  We propose a Diffusion-based 3D Vision Transformer (Diff3Dformer) to aggregate repetitive information within 3D CT scans. Our method exhibits improved performance on two different scales of small datasets of 3D lung CT scans.
  arXiv  Detail & Related papers  (2024-06-24T23:23:18Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows [16.928404625892625]
  Real-time 3D fluorescence microscopy is crucial for the analysis of live organisms. We propose a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity.
  arXiv  Detail & Related papers  (2023-06-10T10:42:49Z)
• Image Reconstruction for Accelerated MR Scan with Faster Fourier Convolutional Neural Networks [87.87578529398019]
  Partial scan is a common approach to accelerate Magnetic Resonance Imaging (MRI) data acquisition in both 2D and 3D settings. We propose a novel convolutional operator called Faster Fourier Convolution (FasterFC) to replace the two consecutive convolution operations. A 2D accelerated MRI method, FasterFC-End-to-End-VarNet, which uses FasterFC to improve the sensitivity maps and reconstruction quality. A 3D accelerated MRI method called FasterFC-based Single-to-group Network (FAS-Net) that utilizes a single-to-group algorithm to guide k-space domain reconstruction
  arXiv  Detail & Related papers  (2023-06-05T13:53:57Z)
• View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
  We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection. First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan. Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view. Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
  arXiv  Detail & Related papers  (2022-09-20T11:58:23Z)
• Deep learning of multi-resolution X-Ray micro-CT images for multi-scale modelling [0.0]
  We develop a 3D Enhanced Deep Super Resolution (EDSR) convolutional neural network to create enhanced, high-resolution data over large spatial scales. We validate the network with various metrics: textual analysis, segmentation behaviour and pore-network model (PNM) multiphase flow simulations. The EDSR generated model is more accurate than the base LR model at predicting experimental behaviour in the presence of heterogeneities.
  arXiv  Detail & Related papers  (2021-11-01T21:49:22Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• 3D Solid Spherical Bispectrum CNNs for Biomedical Texture Analysis [3.579867431007686]
  Locally Rotation Invariant (LRI) operators have shown great potential in biomedical texture analysis. We investigate the benefits of using the bispectrum over the spectrum in the design of a LRI layer embedded in a shallow Convolutional Neural Network (CNN) for 3D image analysis.
  arXiv  Detail & Related papers  (2020-04-28T09:01:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.