VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers

• URL: http://arxiv.org/abs/2104.06757v1
• Date: Wed, 14 Apr 2021 10:32:36 GMT
• Title: VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers
• Authors: Sharif Amit Kamran, Khondker Fariha Hossain, Alireza Tavakkoli, Stewart Lee Zuckerbrod, Kenton M. Sanders, Salah A. Baker
• Abstract summary: In Fluorescein Angiography (FA), an injected dye is injected in the bloodstream to image the vascular structure of the retina. Fundus imaging is a non-invasive technique used for photographing the retina but does not have sufficient fidelity for capturing its vascular structure. We propose a novel conditional generative adversarial network (GAN) capable of simultaneously synthesizing FA images from fundus photographs while predicting retinal degeneration.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In Fluorescein Angiography (FA), an exogenous dye is injected in the bloodstream to image the vascular structure of the retina. The injected dye can cause adverse reactions such as nausea, vomiting, anaphylactic shock, and even death. In contrast, color fundus imaging is a non-invasive technique used for photographing the retina but does not have sufficient fidelity for capturing its vascular structure. The only non-invasive method for capturing retinal vasculature is optical coherence tomography-angiography (OCTA). However, OCTA equipment is quite expensive, and stable imaging is limited to small areas on the retina. In this paper, we propose a novel conditional generative adversarial network (GAN) capable of simultaneously synthesizing FA images from fundus photographs while predicting retinal degeneration. The proposed system has the benefit of addressing the problem of imaging retinal vasculature in a non-invasive manner as well as predicting the existence of retinal abnormalities. We use a semi-supervised approach to train our GAN using multiple weighted losses on different modalities of data. Our experiments validate that the proposed architecture exceeds recent state-of-the-art generative networks for fundus-to-angiography synthesis. Moreover, our vision transformer-based discriminators generalize quite well on out-of-distribution data sets for retinal disease prediction.

Related papers

• CUNSB-RFIE: Context-aware Unpaired Neural Schr"{o}dinger Bridge in Retinal Fundus Image Enhancement [15.399449331371402]
  We propose an image-to-image translation pipeline for retinal image enhancement. We name the resulting retinal fundus image enhancement framework the Context-aware Unpaired Neural Schr"odinger Bridge (CUNSB-RFIE)
  arXiv  Detail & Related papers  (2024-09-17T08:07:29Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Synthetic optical coherence tomography angiographs for detailed retinal vessel segmentation without human annotations [12.571349114534597]
  We present a lightweight simulation of the retinal vascular network based on space colonization for faster and more realistic OCTA synthesis. We demonstrate the superior segmentation performance of our approach in extensive quantitative and qualitative experiments on three public datasets.
  arXiv  Detail & Related papers  (2023-06-19T14:01:47Z)
• Deep learning network to correct axial and coronal eye motion in 3D OCT retinal imaging [65.47834983591957]
  We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single scan. The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods.
  arXiv  Detail & Related papers  (2023-05-27T03:55:19Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• NuI-Go: Recursive Non-Local Encoder-Decoder Network for Retinal Image Non-Uniform Illumination Removal [96.12120000492962]
  The quality of retinal images is often clinically unsatisfactory due to eye lesions and imperfect imaging process. One of the most challenging quality degradation issues in retinal images is non-uniform illumination. We propose a non-uniform illumination removal network for retinal image, called NuI-Go.
  arXiv  Detail & Related papers  (2020-08-07T04:31:33Z)
• Attention2AngioGAN: Synthesizing Fluorescein Angiography from Retinal Fundus Images using Generative Adversarial Networks [0.0]
  Fluorescein Angiography (FA) is a technique that employs the designated camera for Fundus photography incorporating excitation and barrier filters. FA also requires fluorescein dye that is injected intravenously, which might cause adverse effects ranging from nausea, vomiting to even fatal anaphylaxis. We introduce an Attention-based Generative network that can synthesize Fluorescein Angiography from Fundus images.
  arXiv  Detail & Related papers  (2020-07-17T18:58:44Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)
• Fundus2Angio: A Conditional GAN Architecture for Generating Fluorescein Angiography Images from Retinal Fundus Photography [0.0]
  There are no non-invasive systems capable of generating Fluorescein Angiography images. Fundus photography is a non-invasive imaging technique that can be completed in a few seconds. We propose a conditional generative adversarial network (GAN) to translate fundus images to FA images.
  arXiv  Detail & Related papers  (2020-05-11T17:09:29Z)
• Retinopathy of Prematurity Stage Diagnosis Using Object Segmentation and Convolutional Neural Networks [68.96150598294072]
  Retinopathy of Prematurity (ROP) is an eye disorder primarily affecting premature infants with lower weights. It causes proliferation of vessels in the retina and could result in vision loss and, eventually, retinal detachment, leading to blindness. In recent years, there has been a significant effort to automate the diagnosis using deep learning. This paper builds upon the success of previous models and develops a novel architecture, which combines object segmentation and convolutional neural networks (CNN) Our proposed system first trains an object segmentation model to identify the demarcation line at a pixel level and adds the resulting mask as an additional "color" channel in
  arXiv  Detail & Related papers  (2020-04-03T14:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.