RetinaRegNet: A Zero-Shot Approach for Retinal Image Registration

• URL: http://arxiv.org/abs/2404.16017v3
• Date: Wed, 11 Sep 2024 00:25:37 GMT
• Title: RetinaRegNet: A Zero-Shot Approach for Retinal Image Registration
• Authors: Vishal Balaji Sivaraman, Muhammad Imran, Qingyue Wei, Preethika Muralidharan, Michelle R. Tamplin, Isabella M . Grumbach, Randy H. Kardon, Jui-Kai Wang, Yuyin Zhou, Wei Shao,
• Abstract summary: RetinaRegNet is a zero-shot registration model designed to register retinal images with minimal overlap, large deformations, and varying image quality. We implement a two-stage registration framework to handle large deformations. Our model consistently outperformed state-of-the-art methods across all datasets.
• Score: 10.430563602981705
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce RetinaRegNet, a zero-shot image registration model designed to register retinal images with minimal overlap, large deformations, and varying image quality. RetinaRegNet addresses these challenges and achieves robust and accurate registration through the following steps. First, we extract features from the moving and fixed images using latent diffusion models. We then sample feature points from the fixed image using a combination of the SIFT algorithm and random point sampling. For each sampled point, we identify its corresponding point in the moving image using a 2D correlation map, which computes the cosine similarity between the diffusion feature vectors of the point in the fixed image and all pixels in the moving image. Second, we eliminate most incorrectly detected point correspondences (outliers) by enforcing an inverse consistency constraint, ensuring that correspondences are consistent in both forward and backward directions. We further remove outliers with large distances between corresponding points using a global transformation based outlier detector. Finally, we implement a two-stage registration framework to handle large deformations. The first stage estimates a homography transformation to achieve global alignment between the images, while the second stage uses a third-order polynomial transformation to estimate local deformations. We evaluated RetinaRegNet on three retinal image registration datasets: color fundus images, fluorescein angiography images, and laser speckle flowgraphy images. Our model consistently outperformed state-of-the-art methods across all datasets. The accurate registration achieved by RetinaRegNet enables the tracking of eye disease progression, enhances surgical planning, and facilitates the evaluation of treatment efficacy. Our code is publicly available at: https://github.com/mirthAI/RetinaRegNet.

Related papers

• Progressive Retinal Image Registration via Global and Local Deformable Transformations [49.032894312826244]
  We propose a hybrid registration framework called HybridRetina. We use a keypoint detector and a deformation network called GAMorph to estimate the global transformation and local deformable transformation. Experiments on two widely-used datasets, FIRE and FLoRI21, show that our proposed HybridRetina significantly outperforms some state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-02T08:43:50Z)
• Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers [50.576354045312115]
  Direct image-to-graph transformation is a challenging task that solves object detection and relationship prediction in a single model. We introduce a set of methods enabling cross-domain and cross-dimension transfer learning for image-to-graph transformers. We demonstrate our method's utility in cross-domain and cross-dimension experiments, where we pretrain our models on 2D satellite images before applying them to vastly different target domains in 2D and 3D.
  arXiv  Detail & Related papers  (2024-03-11T10:48:56Z)
• 2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds [38.425876064671435]
  We propose 2D3D-MATR, a detection-free method for accurate and robust registration between images and point clouds. Our method adopts a coarse-to-fine pipeline where it first computes coarse correspondences between downsampled patches of the input image and the point cloud. To resolve the scale ambiguity in patch matching, we construct a multi-scale pyramid for each image patch and learn to find for each point patch the best matching image patch at a proper resolution level.
  arXiv  Detail & Related papers  (2023-08-10T16:10:54Z)
• Estimating Extreme 3D Image Rotation with Transformer Cross-Attention [13.82735766201496]
  We propose a cross-attention-based approach that utilizes CNN feature maps and a Transformer-Encoder to compute the cross-attention between the activation maps of the image pairs. It is experimentally shown to outperform contemporary state-of-the-art schemes when applied to commonly used image rotation datasets and benchmarks.
  arXiv  Detail & Related papers  (2023-03-05T09:07:26Z)
• Differentiable Uncalibrated Imaging [25.67247660827913]
  We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles. We apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates. Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration.
  arXiv  Detail & Related papers  (2022-11-18T22:48:09Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Unsupervised Domain Adaptation with Contrastive Learning for OCT Segmentation [49.59567529191423]
  We propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains. We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nearby slices in 3D.
  arXiv  Detail & Related papers  (2022-03-07T19:02:26Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• DeepI2P: Image-to-Point Cloud Registration via Deep Classification [71.3121124994105]
  DeepI2P is a novel approach for cross-modality registration between an image and a point cloud. Our method estimates the relative rigid transformation between the coordinate frames of the camera and Lidar. We circumvent the difficulty by converting the registration problem into a classification and inverse camera projection optimization problem.
  arXiv  Detail & Related papers  (2021-04-08T04:27:32Z)
• FlowReg: Fast Deformable Unsupervised Medical Image Registration using Optical Flow [0.09167082845109438]
  FlowReg is a framework for unsupervised image registration for neuroimaging applications. FlowReg is able to obtain high intensity and spatial similarity while maintaining the shape and structure of anatomy and pathology.
  arXiv  Detail & Related papers  (2021-01-24T03:51:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.