OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to
Confocal Microscopy
- URL: http://arxiv.org/abs/2311.10902v3
- Date: Sat, 17 Feb 2024 01:38:22 GMT
- Title: OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to
Confocal Microscopy
- Authors: Xin Tian, Nantheera Anantrasirichai, Lindsay Nicholson, Alin Achim
- Abstract summary: We develop a 3D CycleGAN framework for unsupervised translation of in-vivo OCT to ex-vivo confocal microscopy images.
This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy.
- Score: 12.367828307288105
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Optical coherence tomography (OCT) and confocal microscopy are pivotal in
retinal imaging, each presenting unique benefits and limitations. In-vivo OCT
offers rapid, non-invasive imaging but can be hampered by clarity issues and
motion artifacts. Ex-vivo confocal microscopy provides high-resolution,
cellular detailed color images but is invasive and poses ethical concerns and
potential tissue damage. To bridge these modalities, we developed a 3D CycleGAN
framework for unsupervised translation of in-vivo OCT to ex-vivo confocal
microscopy images. Applied to our OCT2Confocal dataset, this framework
effectively translates between 3D medical data domains, capturing vascular,
textural, and cellular details with precision. This marks the first attempt to
exploit the inherent 3D information of OCT and translate it into the rich,
detailed color domain of confocal microscopy. Assessed through quantitative and
qualitative evaluations, the 3D CycleGAN framework demonstrates commendable
image fidelity and quality, outperforming existing methods despite the
constraints of limited data. This non-invasive generation of retinal confocal
images has the potential to further enhance diagnostic and monitoring
capabilities in ophthalmology. Our source code and OCT2Confocal dataset are
available at https://github.com/xintian-99/OCT2Confocal.
Related papers
- DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays [41.393567374399524]
We propose DiffuX2CT, which models CT reconstruction from ultra-sparse X-rays as a conditional diffusion process.
By doing so, DiffuX2CT achieves structure-controllable reconstruction, which enables 3D structural information to be recovered from 2D X-rays.
As an extra contribution, we collect a real-world lumbar CT dataset, called LumbarV, as a new benchmark to verify the clinical significance and performance of CT reconstruction from X-rays.
arXiv Detail & Related papers (2024-07-18T14:20:04Z) - μ-Net: A Deep Learning-Based Architecture for μ-CT Segmentation [2.012378666405002]
X-ray computed microtomography (mu-CT) is a non-destructive technique that can generate high-resolution 3D images of the internal anatomy of medical and biological samples.
extracting relevant information from 3D images requires semantic segmentation of the regions of interest.
We propose a novel framework that uses a convolutional neural network (CNN) to automatically segment the full morphology of the heart of Carassius auratus.
arXiv Detail & Related papers (2024-06-24T15:29:08Z) - CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning.
Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
arXiv Detail & Related papers (2024-04-23T17:59:01Z) - Super-resolution of biomedical volumes with 2D supervision [84.5255884646906]
Masked slice diffusion for super-resolution exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens.
We focus on the application of SliceR to stimulated histology (SRH), characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning.
arXiv Detail & Related papers (2024-04-15T02:41:55Z) - Multi-View Vertebra Localization and Identification from CT Images [57.56509107412658]
We propose a multi-view vertebra localization and identification from CT images.
We convert the 3D problem into a 2D localization and identification task on different views.
Our method can learn the multi-view global information naturally.
arXiv Detail & Related papers (2023-07-24T14:43:07Z) - Deep learning network to correct axial and coronal eye motion in 3D OCT
retinal imaging [65.47834983591957]
We propose deep learning based neural networks to correct axial and coronal motion artifacts in OCT based on a single scan.
The experimental result shows that the proposed method can effectively correct motion artifacts and achieve smaller error than other methods.
arXiv Detail & Related papers (2023-05-27T03:55:19Z) - Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual
Constraints [27.28468771485413]
We propose a novel framework, TransPro, that translates 3D Optical Coherence Tomography ( OCT) images into exclusive 3D OCTA images using an image translation pattern.
Our main objective is to address two issues in existing image translation baselines, namely, the aimlessness in the translation process and incompleteness of the translated object.
TransPro consistently outperforms existing approaches with minimal computational overhead during training and none during testing.
arXiv Detail & Related papers (2023-03-13T01:42:29Z) - nnUNet RASPP for Retinal OCT Fluid Detection, Segmentation and
Generalisation over Variations of Data Sources [25.095695898777656]
We propose two variants of the nnUNet with consistent high performance across images from multiple device vendors.
The algorithm was validated on the MICCAI 2017 RETOUCH challenge dataset.
Experimental results show that our algorithms outperform the current state-of-the-arts algorithms.
arXiv Detail & Related papers (2023-02-25T23:47:23Z) - Affinity Feature Strengthening for Accurate, Complete and Robust Vessel
Segmentation [48.638327652506284]
Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms.
We present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach.
arXiv Detail & Related papers (2022-11-12T05:39:17Z) - Medical Diffusion -- Denoising Diffusion Probabilistic Models for 3D
Medical Image Generation [0.6486409713123691]
We show that diffusion probabilistic models can synthesize high quality medical imaging data.
We provide quantitative measurements of their performance through a reader study with two medical experts.
We demonstrate that synthetic images can be used in a self-supervised pre-training and improve the performance of breast segmentation models when data is scarce.
arXiv Detail & Related papers (2022-11-07T08:37:48Z) - Automated Model Design and Benchmarking of 3D Deep Learning Models for
COVID-19 Detection with Chest CT Scans [72.04652116817238]
We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification.
We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
arXiv Detail & Related papers (2021-01-14T03:45:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.