Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation

• URL: http://arxiv.org/abs/2204.13656v1
• Date: Thu, 28 Apr 2022 17:18:21 GMT
• Title: Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation
• Authors: Zekang Chen, Jia Wei and Rui Li
• Abstract summary: We propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes.
• Score: 4.43142018105102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In clinical practice, well-aligned multi-modal images, such as Magnetic Resonance (MR) and Computed Tomography (CT), together can provide complementary information for image-guided therapies. Multi-modal image registration is essential for the accurate alignment of these multi-modal images. However, it remains a very challenging task due to complicated and unknown spatial correspondence between different modalities. In this paper, we propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Specifically, our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes. Furthermore, we propose to replace an adversarial loss, that is widely used in previous multi-modal image registration methods, with a pixel loss in order to integrate the output of translation into the target modality. This leads to an unsupervised method requiring no ground-truth deformation or pairs of aligned images for training. We evaluate four variants of our approach on the public Learn2Reg 2021 datasets \cite{hering2021learn2reg}. The experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. Our code is available at https://github.com/heyblackC/DFMIR.

Related papers

• Large Language Models for Multimodal Deformable Image Registration [50.91473745610945]
  We propose a novel coarse-to-fine MDIR framework,LLM-Morph, for aligning the deep features from different modal medical images. Specifically, we first utilize a CNN encoder to extract deep visual features from cross-modal image pairs, then we use the first adapter to adjust these tokens, and use LoRA in pre-trained LLMs to fine-tune their weights. Third, for the alignment of tokens, we utilize other four adapters to transform the LLM-encoded tokens into multi-scale visual features, generating multi-scale deformation fields and facilitating the coarse-to-fine MDIR task
  arXiv  Detail & Related papers  (2024-08-20T09:58:30Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• Multi-modal unsupervised brain image registration using edge maps [7.49320945341034]
  We propose a simple yet effective unsupervised deep learning-based em multi-modal image registration approach. The intuition behind this is that image locations with a strong gradient are assumed to denote a transition of tissues. We evaluate our approach in the context of registering multi-modal (T1w to T2w) magnetic resonance (MR) brain images of different subjects using three different loss functions.
  arXiv  Detail & Related papers  (2022-02-09T15:50:14Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis [68.3787368024951]
  We propose a novel approach for multi-modal Image-to-image (I2I) translation. We learn a latent embedding, jointly with the generator, that models the variability of the output domain. Specifically, we pre-train a generic style encoder using a novel proxy task to learn an embedding of images, from arbitrary domains, into a low-dimensional style latent space.
  arXiv  Detail & Related papers  (2021-04-14T19:58:24Z)
• Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study [4.00906288611816]
  We conduct an empirical study of the applicability of modern I2I translation methods for the task of multimodal biomedical image registration. We compare the performance of four Generative Adrial Network (GAN)-based methods and one contrastive representation learning method. Our results suggest that, although I2I translation may be helpful when the modalities to register are clearly correlated, registration of modalities which express distinctly different properties of the sample are not well handled by the I2I translation approach.
  arXiv  Detail & Related papers  (2021-03-30T11:28:21Z)
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation [9.659642285903418]
  Cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups.
  arXiv  Detail & Related papers  (2021-03-05T16:22:31Z)
• Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration [20.637787406888478]
  Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.
  arXiv  Detail & Related papers  (2020-07-06T14:44:06Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)
• Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation [43.060971647266236]
  We train an image-to-image translation network on the two input modalities. This learned translation allows training the registration network using simple and reliable mono-modality metrics. Compared to state-of-the-art multi-modal methods our presented method is unsupervised, requiring no pairs of aligned modalities for training, and can be adapted to any pair of modalities.
  arXiv  Detail & Related papers  (2020-03-18T07:21:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.