A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration

• URL: http://arxiv.org/abs/2507.18551v1
• Date: Thu, 24 Jul 2025 16:19:08 GMT
• Title: A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
• Authors: Daniil Morozov, Reuben Dorent, Nazim Haouchine,
• Abstract summary: Intraoperative registration of real-time ultrasound to preoperative Magnetic Resonance Imaging (MRI) remains an unsolved problem.<n>We propose a novel 3D cross-modal keypoint descriptor for MRI-iUS matching and registration.<n>Our approach employs a patient-specific matching-by-synthesis approach, generating synthetic iUS volumes from preoperative MRI.
• Score: 0.053801353100098995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intraoperative registration of real-time ultrasound (iUS) to preoperative Magnetic Resonance Imaging (MRI) remains an unsolved problem due to severe modality-specific differences in appearance, resolution, and field-of-view. To address this, we propose a novel 3D cross-modal keypoint descriptor for MRI-iUS matching and registration. Our approach employs a patient-specific matching-by-synthesis approach, generating synthetic iUS volumes from preoperative MRI. This enables supervised contrastive training to learn a shared descriptor space. A probabilistic keypoint detection strategy is then employed to identify anatomically salient and modality-consistent locations. During training, a curriculum-based triplet loss with dynamic hard negative mining is used to learn descriptors that are i) robust to iUS artifacts such as speckle noise and limited coverage, and ii) rotation-invariant . At inference, the method detects keypoints in MR and real iUS images and identifies sparse matches, which are then used to perform rigid registration. Our approach is evaluated using 3D MRI-iUS pairs from the ReMIND dataset. Experiments show that our approach outperforms state-of-the-art keypoint matching methods across 11 patients, with an average precision of $69.8\%$. For image registration, our method achieves a competitive mean Target Registration Error of 2.39 mm on the ReMIND2Reg benchmark. Compared to existing iUS-MR registration approach, our framework is interpretable, requires no manual initialization, and shows robustness to iUS field-of-view variation. Code is available at https://github.com/morozovdd/CrossKEY.

Related papers

• ContextMRI: Enhancing Compressed Sensing MRI through Metadata Conditioning [51.26601171361753]
  We propose ContextMRI, a text-conditioned diffusion model for MRI that integrates granular metadata into the reconstruction process.<n>We show that increasing the fidelity of metadata, ranging from slice location and contrast to patient age, sex, and pathology, systematically boosts reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-08T05:15:43Z)
• CABLD: Contrast-Agnostic Brain Landmark Detection with Consistency-Based Regularization [2.423045468361048]
  We introduce CABLD, a novel self-supervised deep learning framework for 3D brain landmark detection in unlabeled scans.<n>We demonstrate the proposed method with the intricate task of MRI-based 3D brain landmark detection.<n>Our framework provides a robust and accurate solution for anatomical landmark detection, reducing the need for extensively annotated datasets.
  arXiv  Detail & Related papers  (2024-11-26T19:56:29Z)
• Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound [38.1299082729891]
  We propose a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.
  arXiv  Detail & Related papers  (2024-09-12T16:00:22Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Explainable unsupervised multi-modal image registration using deep networks [2.197364252030876]
  MRI image registration aims to geometrically 'pair' diagnoses from different modalities, time points and slices. In this work, we show that our DL model becomes fully explainable, setting the framework to generalise our approach on further medical imaging data.
  arXiv  Detail & Related papers  (2023-08-03T19:13:48Z)
• GSMorph: Gradient Surgery for cine-MRI Cardiac Deformable Registration [62.41725951450803]
  Learning-based deformable registration relies on weighted objective functions trading off registration accuracy and smoothness of the field. We construct a registration model based on the gradient surgery mechanism, named GSMorph, to achieve a hyper parameter-free balance on multiple losses. Our method is model-agnostic and can be merged into any deep registration network without introducing extra parameters or slowing down inference.
  arXiv  Detail & Related papers  (2023-06-26T13:32:09Z)
• Meta-Learning Initializations for Interactive Medical Image Registration [0.18750851274087482]
  This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application. Applying sparsely sampled data to non-interactive methods yields higher registration errors (6.26 mm), demonstrating the effectiveness of interactive MR-TRUS registration.
  arXiv  Detail & Related papers  (2022-10-27T12:30:53Z)
• Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
  We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
  arXiv  Detail & Related papers  (2022-09-19T09:43:19Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)
• Deep Learning based Multi-modal Computing with Feature Disentanglement for MRI Image Synthesis [8.363448006582065]
  We propose a deep learning based multi-modal computing model for MRI synthesis with feature disentanglement strategy. The proposed approach decomposes each input modality into modality-invariant space with shared information and modality-specific space with specific information. To address the lack of specific information of the target modality in the test phase, a local adaptive fusion (LAF) module is adopted to generate a modality-like pseudo-target.
  arXiv  Detail & Related papers  (2021-05-06T17:22:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.