Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual
Constraints
- URL: http://arxiv.org/abs/2303.06807v1
- Date: Mon, 13 Mar 2023 01:42:29 GMT
- Title: Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual
Constraints
- Authors: Shuhan Li, Dong Zhang, Xiaomeng Li, Chubin Ou, Lin An, Yanwu Xu,
Kwang-Ting Cheng
- Abstract summary: We propose a novel framework, TransPro, that translates 3D Optical Coherence Tomography ( OCT) images into exclusive 3D OCTA images using an image translation pattern.
Our main objective is to address two issues in existing image translation baselines, namely, the aimlessness in the translation process and incompleteness of the translated object.
TransPro consistently outperforms existing approaches with minimal computational overhead during training and none during testing.
- Score: 27.28468771485413
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Optical Coherence Tomography Angiography (OCTA) has become increasingly vital
in the clinical screening of fundus diseases due to its ability to capture
accurate 3D imaging of blood vessels in a non-contact scanning manner. However,
the acquisition of OCTA images remains challenging due to the requirement of
exclusive sensors and expensive devices. In this paper, we propose a novel
framework, TransPro, that translates 3D Optical Coherence Tomography (OCT)
images into exclusive 3D OCTA images using an image translation pattern. Our
main objective is to address two issues in existing image translation
baselines, namely, the aimlessness in the translation process and
incompleteness of the translated object. The former refers to the overall
quality of the translated OCTA images being satisfactory, but the retinal
vascular quality being low. The latter refers to incomplete objects in
translated OCTA images due to the lack of global contexts. TransPro merges a 2D
retinal vascular segmentation model and a 2D OCTA image translation model into
a 3D image translation baseline for the 2D projection map projected by the
translated OCTA images. The 2D retinal vascular segmentation model can improve
attention to the retinal vascular, while the 2D OCTA image translation model
introduces beneficial heuristic contextual information. Extensive experimental
results on two challenging datasets demonstrate that TransPro can consistently
outperform existing approaches with minimal computational overhead during
training and none during testing.
Related papers
- Diff3Dformer: Leveraging Slice Sequence Diffusion for Enhanced 3D CT Classification with Transformer Networks [5.806035963947936]
We propose a Diffusion-based 3D Vision Transformer (Diff3Dformer) to aggregate repetitive information within 3D CT scans.
Our method exhibits improved performance on two different scales of small datasets of 3D lung CT scans.
arXiv Detail & Related papers (2024-06-24T23:23:18Z) - 3D Vessel Reconstruction from Sparse-View Dynamic DSA Images via Vessel Probability Guided Attenuation Learning [79.60829508459753]
Current commercial Digital Subtraction Angiography (DSA) systems typically demand hundreds of scanning views to perform reconstruction.
The dynamic blood flow and insufficient input of sparse-view DSA images present significant challenges to the 3D vessel reconstruction task.
We propose to use a time-agnostic vessel probability field to solve this problem effectively.
arXiv Detail & Related papers (2024-05-17T11:23:33Z) - CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning.
Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
arXiv Detail & Related papers (2024-04-23T17:59:01Z) - DISCOVER: 2-D Multiview Summarization of Optical Coherence Tomography
Angiography for Automatic Diabetic Retinopathy Diagnosis [1.6788804044046786]
Diabetic Retinopathy (DR), an ocular complication of diabetes, is a leading cause of blindness worldwide.
Traditionally, DR is monitored using Color Fundus Photography (CFP), a widespread 2-D imaging modality.
This paper investigates automatic DR assessment using 3-D OCTA.
arXiv Detail & Related papers (2024-01-10T13:06:40Z) - Deep Learning for Vascular Segmentation and Applications in Phase
Contrast Tomography Imaging [33.23991248643144]
We present a thorough literature review, highlighting the state of machine learning techniques across diverse organs.
Our goal is to provide a foundation on the topic and identify a robust baseline model for application to vascular segmentation in a new imaging modality.
HiP CT enables 3D imaging of complete organs at an unprecedented resolution of ca. 20mm per voxel.
arXiv Detail & Related papers (2023-11-22T11:15:38Z) - OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to
Confocal Microscopy [12.367828307288105]
We develop a 3D CycleGAN framework for unsupervised translation of in-vivo OCT to ex-vivo confocal microscopy images.
This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy.
arXiv Detail & Related papers (2023-11-17T22:48:50Z) - Homography Loss for Monocular 3D Object Detection [54.04870007473932]
A differentiable loss function, termed as Homography Loss, is proposed to achieve the goal, which exploits both 2D and 3D information.
Our method yields the best performance compared with the other state-of-the-arts by a large margin on KITTI 3D datasets.
arXiv Detail & Related papers (2022-04-02T03:48:03Z) - CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for
Non-Contrast to Contrast CT Translation [56.622832383316215]
We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans.
Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran.
Our empirical results show that CyTran outperforms all competing methods.
arXiv Detail & Related papers (2021-10-12T23:25:03Z) - Uncertainty-Guided Progressive GANs for Medical Image Translation [37.95176881950121]
Image-to-image translation plays a vital role in tackling various medical imaging tasks.
We propose an uncertainty-guided progressive learning scheme for image-to-image translation.
We demonstrate the efficacy of our model on three challenging medical image translation tasks.
arXiv Detail & Related papers (2021-06-29T16:26:12Z) - Revisiting 3D Context Modeling with Supervised Pre-training for
Universal Lesion Detection in CT Slices [48.85784310158493]
We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices.
With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset.
The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
arXiv Detail & Related papers (2020-12-16T07:11:16Z) - Self-Supervised 2D Image to 3D Shape Translation with Disentangled
Representations [92.89846887298852]
We present a framework to translate between 2D image views and 3D object shapes.
We propose SIST, a Self-supervised Image to Shape Translation framework.
arXiv Detail & Related papers (2020-03-22T22:44:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.