OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to Confocal Microscopy

• URL: http://arxiv.org/abs/2311.10902v3
• Date: Sat, 17 Feb 2024 01:38:22 GMT
• Title: OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to Confocal Microscopy
• Authors: Xin Tian, Nantheera Anantrasirichai, Lindsay Nicholson, Alin Achim
• Abstract summary: We develop a 3D CycleGAN framework for unsupervised translation of in-vivo OCT to ex-vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy.
• Score: 12.367828307288105
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Optical coherence tomography (OCT) and confocal microscopy are pivotal in retinal imaging, each presenting unique benefits and limitations. In-vivo OCT offers rapid, non-invasive imaging but can be hampered by clarity issues and motion artifacts. Ex-vivo confocal microscopy provides high-resolution, cellular detailed color images but is invasive and poses ethical concerns and potential tissue damage. To bridge these modalities, we developed a 3D CycleGAN framework for unsupervised translation of in-vivo OCT to ex-vivo confocal microscopy images. Applied to our OCT2Confocal dataset, this framework effectively translates between 3D medical data domains, capturing vascular, textural, and cellular details with precision. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy. Assessed through quantitative and qualitative evaluations, the 3D CycleGAN framework demonstrates commendable image fidelity and quality, outperforming existing methods despite the constraints of limited data. This non-invasive generation of retinal confocal images has the potential to further enhance diagnostic and monitoring capabilities in ophthalmology. Our source code and OCT2Confocal dataset are available at https://github.com/xintian-99/OCT2Confocal.

Related papers

• 3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models [51.855377054763345]
  This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model for generating radiology reports from 3D CT scans. Experiments on both public and private datasets demonstrate that 3D-CT-GPT significantly outperforms existing methods in terms of report accuracy and quality.
  arXiv  Detail & Related papers  (2024-09-28T12:31:07Z)
• OCTCube: A 3D foundation model for optical coherence tomography that improves cross-dataset, cross-disease, cross-device and cross-modality analysis [11.346324975034051]
  OCTCube is a 3D foundation model pre-trained on 26,605 3D OCT volumes encompassing 1.62 million 2D OCT images. It outperforms 2D models when predicting 8 retinal diseases in both inductive and cross-dataset settings. It also shows superior performance on cross-device prediction and when predicting systemic diseases, such as diabetes and hypertension.
  arXiv  Detail & Related papers  (2024-08-20T22:55:19Z)
• The Quest for Early Detection of Retinal Disease: 3D CycleGAN-based Translation of Optical Coherence Tomography into Confocal Microscopy [11.321411104729002]
  We propose a novel framework based on unsupervised 3D CycleGAN for translating unpaired in vivo OCT to ex vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy.
  arXiv  Detail & Related papers  (2024-08-07T21:13:49Z)
• DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays [41.393567374399524]
  We propose DiffuX2CT, which models CT reconstruction from ultra-sparse X-rays as a conditional diffusion process. By doing so, DiffuX2CT achieves structure-controllable reconstruction, which enables 3D structural information to be recovered from 2D X-rays. As an extra contribution, we collect a real-world lumbar CT dataset, called LumbarV, as a new benchmark to verify the clinical significance and performance of CT reconstruction from X-rays.
  arXiv  Detail & Related papers  (2024-07-18T14:20:04Z)
• μ-Net: A Deep Learning-Based Architecture for μ-CT Segmentation [2.012378666405002]
  X-ray computed microtomography (mu-CT) is a non-destructive technique that can generate high-resolution 3D images of the internal anatomy of medical and biological samples. extracting relevant information from 3D images requires semantic segmentation of the regions of interest. We propose a novel framework that uses a convolutional neural network (CNN) to automatically segment the full morphology of the heart of Carassius auratus.
  arXiv  Detail & Related papers  (2024-06-24T15:29:08Z)
• CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
  We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning. Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
  arXiv  Detail & Related papers  (2024-04-23T17:59:01Z)
• Super-resolution of biomedical volumes with 2D supervision [84.5255884646906]
  Masked slice diffusion for super-resolution exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens. We focus on the application of SliceR to stimulated histology (SRH), characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning.
  arXiv  Detail & Related papers  (2024-04-15T02:41:55Z)
• Multi-View Vertebra Localization and Identification from CT Images [57.56509107412658]
  We propose a multi-view vertebra localization and identification from CT images. We convert the 3D problem into a 2D localization and identification task on different views. Our method can learn the multi-view global information naturally.
  arXiv  Detail & Related papers  (2023-07-24T14:43:07Z)
• Deep learning network to correct axial and coronal eye motion in 3D OCT retinal imaging [65.47834983591957]
  We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single scan. The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods.
  arXiv  Detail & Related papers  (2023-05-27T03:55:19Z)
• nnUNet RASPP for Retinal OCT Fluid Detection, Segmentation and Generalisation over Variations of Data Sources [25.095695898777656]
  We propose two variants of the nnUNet with consistent high performance across images from multiple device vendors. The algorithm was validated on the MICCAI 2017 RETOUCH challenge dataset. Experimental results show that our algorithms outperform the current state-of-the-arts algorithms.
  arXiv  Detail & Related papers  (2023-02-25T23:47:23Z)
• Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans [72.04652116817238]
  We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification. We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
  arXiv  Detail & Related papers  (2021-01-14T03:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.