SAMReg: SAM-enabled Image Registration with ROI-based Correspondence

• URL: http://arxiv.org/abs/2410.14083v1
• Date: Thu, 17 Oct 2024 23:23:48 GMT
• Title: SAMReg: SAM-enabled Image Registration with ROI-based Correspondence
• Authors: Shiqi Huang, Tingfa Xu, Ziyi Shen, Shaheer Ullah Saeed, Wen Yan, Dean Barratt, Yipeng Hu,
• Abstract summary: This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs) for medical image registration. We develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics.
• Score: 12.163299991979574
• License:
• Abstract: This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs), for medical image registration. The distinct properties of the proposed ROI-based correspondence are discussed, in the context of potential benefits in clinical applications following image registration, compared with alternative correspondence-representing approaches, such as those based on sampled displacements and spatial transformation functions. These benefits include a clear connection between learning-based image registration and segmentation, which in turn motivates two cases of image registration approaches using (pre-)trained segmentation networks. Based on the segment anything model (SAM), a vision foundation model for segmentation, we develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed SAMReg models are evaluated across five real-world applications, including intra-subject registration tasks with cardiac MR and lung CT, challenging inter-subject registration scenarios with prostate MR and retinal imaging, and an additional evaluation with a non-clinical example with aerial image registration. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics including Dice and target registration errors on anatomical structures, and further demonstrates competitive performance compared to weakly-supervised registration approaches that rely on fully-segmented training data. Open source code and examples are available at: https://github.com/sqhuang0103/SAMReg.git.

Related papers

• One registration is worth two segmentations [12.163299991979574]
  The goal of image registration is to establish spatial correspondence between two or more images. We propose an alternative but more intuitive correspondence representation: a set of corresponding regions-of-interest (ROI) pairs. We experimentally show that the proposed SAMReg is capable of segmenting and matching multiple ROI pairs.
  arXiv  Detail & Related papers  (2024-05-17T16:14:32Z)
• Semi-weakly-supervised neural network training for medical image registration [18.520388065729552]
  This paper describes a semi-weakly-supervised registration pipeline that improves the model performance. We examine two types of augmentation methods by perturbation on network weights and image resampling. Experiments using 589 male pelvic MR images, labelled with eight anatomical ROIs, show the improvement in registration performance.
  arXiv  Detail & Related papers  (2024-02-16T14:44:40Z)
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration [50.62725807357586]
  This article presents a general Bayesian learning framework for multi-modal groupwise image registration. We propose a novel hierarchical variational auto-encoding architecture to realise the inference procedure of the latent variables. Experiments were conducted to validate the proposed framework, including four different datasets from cardiac, brain, and abdominal medical images.
  arXiv  Detail & Related papers  (2024-01-04T08:46:39Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Spatial Correspondence between Graph Neural Network-Segmented Images [1.807691213023136]
  Graph neural networks (GNNs) have been proposed for medical image segmentation. This work explores the potentials in these GNNs with common topology for establishing spatial correspondence. With an example application of registering local vertebral sub-regions found in CT images, our experimental results showed that the GNN-based segmentation is capable of accurate and reliable localization.
  arXiv  Detail & Related papers  (2023-03-12T03:25:01Z)
• Recurrence With Correlation Network for Medical Image Registration [66.63200823918429]
  We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets.
  arXiv  Detail & Related papers  (2023-02-05T02:41:46Z)
• Joint segmentation and discontinuity-preserving deformable registration: Application to cardiac cine-MR images [74.99415008543276]
  Most deep learning-based registration methods assume that the deformation fields are smooth and continuous everywhere in the image domain. We propose a novel discontinuity-preserving image registration method to tackle this challenge, which ensures globally discontinuous and locally smooth deformation fields. A co-attention block is proposed in the segmentation component of the network to learn the structural correlations in the input images. We evaluate our method on the task of intra-subject-temporal image registration using large-scale cinematic cardiac magnetic resonance image sequences.
  arXiv  Detail & Related papers  (2022-11-24T23:45:01Z)
• A Deep Discontinuity-Preserving Image Registration Network [73.03885837923599]
  Most deep learning-based registration methods assume that the desired deformation fields are globally smooth and continuous. We propose a weakly-supervised Deep Discontinuity-preserving Image Registration network (DDIR) to obtain better registration performance and realistic deformation fields. We demonstrate that our method achieves significant improvements in registration accuracy and predicts more realistic deformations, in registration experiments on cardiac magnetic resonance (MR) images.
  arXiv  Detail & Related papers  (2021-07-09T13:35:59Z)
• CAR-Net: Unsupervised Co-Attention Guided Registration Network for Joint Registration and Structure Learning [73.03885837923599]
  We propose a novel deep learning network, Co-Attention guided Registration Network (CAR-Net) CAR-Net employs a co-attention block to learn a new representation of the inputs, which drives the registration of the fixed and moving images. Experiments on UK Biobank cardiac cine-magnetic resonance image data demonstrate that CAR-Net obtains higher registration accuracy and smoother deformation fields than state-of-the-art unsupervised registration methods.
  arXiv  Detail & Related papers  (2021-06-11T23:25:49Z)
• MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation [14.36896617430302]
  We propose a new image registration framework based on generative model (MvMM) and neural network estimation. A generative model consolidating both appearance and anatomical information is established to derive a novel loss function capable of implementing groupwise registration. We highlight the versatility of the proposed framework for various applications on multimodal cardiac images.
  arXiv  Detail & Related papers  (2020-06-28T11:19:15Z)
• CoMIR: Contrastive Multimodal Image Representation for Registration [4.543268895439618]
  We propose contrastive coding to learn shared, dense image representations, referred to as CoMIRs (Contrastive Multimodal Image Representations) CoMIRs enable the registration of multimodal images where existing registration methods often fail due to a lack of sufficiently similar image structures.
  arXiv  Detail & Related papers  (2020-06-11T10:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.