Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

• URL: http://arxiv.org/abs/2207.10506v1
• Date: Thu, 21 Jul 2022 14:36:51 GMT
• Title: Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network
• Authors: Aline Sindel, Bettina Hohberger, Andreas Maier, Vincent Christlein
• Abstract summary: We propose an end-to-end trainable deep learning method for multi-modal retinal image registration. Our method extracts convolutional features from the vessel structure for keypoint detection and description. The keypoint detection and description network and graph neural network are jointly trained in a self-supervised manner.
• Score: 9.988115865060589
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In ophthalmological imaging, multiple imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography, are often involved to make a diagnosis of retinal disease. Multi-modal retinal registration techniques can assist ophthalmologists by providing a pixel-based comparison of aligned vessel structures in images from different modalities or acquisition times. To this end, we propose an end-to-end trainable deep learning method for multi-modal retinal image registration. Our method extracts convolutional features from the vessel structure for keypoint detection and description and uses a graph neural network for feature matching. The keypoint detection and description network and graph neural network are jointly trained in a self-supervised manner using synthetic multi-modal image pairs and are guided by synthetically sampled ground truth homographies. Our method demonstrates higher registration accuracy as competing methods for our synthetic retinal dataset and generalizes well for our real macula dataset and a public fundus dataset.

Related papers

• Progressive Retinal Image Registration via Global and Local Deformable Transformations [49.032894312826244]
  We propose a hybrid registration framework called HybridRetina. We use a keypoint detector and a deformation network called GAMorph to estimate the global transformation and local deformable transformation. Experiments on two widely-used datasets, FIRE and FLoRI21, show that our proposed HybridRetina significantly outperforms some state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-02T08:43:50Z)
• TaGAT: Topology-Aware Graph Attention Network For Multi-modal Retinal Image Fusion [11.321411104729002]
  We propose the Topology-Aware Graph Attention Network (TaGAT) for multi-modal retinal image fusion. Our model outperforms state-of-the-art methods in Fluorescein Fundus Angiography (FFA) with Color Fundus (CF) and Optical Coherence Tomography ( OCT) with confocal microscopy retinal image fusion.
  arXiv  Detail & Related papers  (2024-07-19T10:32:06Z)
• Synthetic optical coherence tomography angiographs for detailed retinal vessel segmentation without human annotations [12.571349114534597]
  We present a lightweight simulation of the retinal vascular network based on space colonization for faster and more realistic OCTA synthesis. We demonstrate the superior segmentation performance of our approach in extensive quantitative and qualitative experiments on three public datasets.
  arXiv  Detail & Related papers  (2023-06-19T14:01:47Z)
• Controllable Mind Visual Diffusion Model [58.83896307930354]
  Brain signal visualization has emerged as an active research area, serving as a critical interface between the human visual system and computer vision models. We propose a novel approach, referred to as Controllable Mind Visual Model Diffusion (CMVDM) CMVDM extracts semantic and silhouette information from fMRI data using attribute alignment and assistant networks. We then leverage a control model to fully exploit the extracted information for image synthesis, resulting in generated images that closely resemble the visual stimuli in terms of semantics and silhouette.
  arXiv  Detail & Related papers  (2023-05-17T11:36:40Z)
• Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation [48.638327652506284]
  Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. We present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach.
  arXiv  Detail & Related papers  (2022-11-12T05:39:17Z)
• Segmentation-guided Domain Adaptation and Data Harmonization of Multi-device Retinal Optical Coherence Tomography using Cycle-Consistent Generative Adversarial Networks [2.968191199408213]
  This paper proposes a segmentation-guided domain-adaptation method to adapt images from multiple devices into single image domain. It avoids the time consumption of manual labelling for the upcoming new dataset and the re-training of the existing network.
  arXiv  Detail & Related papers  (2022-08-31T05:06:00Z)
• A novel approach for glaucoma classification by wavelet neural networks using graph-based, statisitcal features of qualitatively improved images [0.0]
  We have proposed a new glaucoma classification approach that employs a wavelet neural network (WNN) on optimally enhanced retinal images features. The performance of the WNN classifier is compared with multilayer perceptron neural networks with various datasets.
  arXiv  Detail & Related papers  (2022-06-24T06:19:30Z)
• A Keypoint Detection and Description Network Based on the Vessel Structure for Multi-Modal Retinal Image Registration [0.0]
  Multiple images with different modalities or acquisition times are often analyzed for the diagnosis of retinal diseases. Our method uses a convolutional neural network to extract features of the vessel structure in multi-modal retinal images.
  arXiv  Detail & Related papers  (2022-01-06T20:43:35Z)
• Semantic segmentation of multispectral photoacoustic images using deep learning [53.65837038435433]
  Photoacoustic imaging has the potential to revolutionise healthcare. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. We present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images.
  arXiv  Detail & Related papers  (2021-05-20T09:33:55Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.