Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis
- URL: http://arxiv.org/abs/2008.08698v1
- Date: Wed, 19 Aug 2020 22:56:36 GMT
- Title: Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis
- Authors: Jianbo Jiao, Ana I.L. Namburete, Aris T. Papageorghiou, J. Alison
Noble
- Abstract summary: Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening.
In this paper we propose to generate MR-like images directly from clinical US images.
The proposed model is end-to-end trainable and self-supervised without any external annotations.
- Score: 20.53251934808636
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the
developing brain but is not suitable for second-trimester anomaly screening,
for which ultrasound (US) is employed. Although expert sonographers are adept
at reading US images, MR images which closely resemble anatomical images are
much easier for non-experts to interpret. Thus in this paper we propose to
generate MR-like images directly from clinical US images. In medical image
analysis such a capability is potentially useful as well, for instance for
automatic US-MRI registration and fusion. The proposed model is end-to-end
trainable and self-supervised without any external annotations. Specifically,
based on an assumption that the US and MRI data share a similar anatomical
latent space, we first utilise a network to extract the shared latent features,
which are then used for MRI synthesis. Since paired data is unavailable for our
study (and rare in practice), pixel-level constraints are infeasible to apply.
We instead propose to enforce the distributions to be statistically
indistinguishable, by adversarial learning in both the image domain and feature
space. To regularise the anatomical structures between US and MRI during
synthesis, we further propose an adversarial structural constraint. A new
cross-modal attention technique is proposed to utilise non-local spatial
information, by encouraging multi-modal knowledge fusion and propagation. We
extend the approach to consider the case where 3D auxiliary information (e.g.,
3D neighbours and a 3D location index) from volumetric data is also available,
and show that this improves image synthesis. The proposed approach is evaluated
quantitatively and qualitatively with comparison to real fetal MR images and
other approaches to synthesis, demonstrating its feasibility of synthesising
realistic MR images.
Related papers
- Synthetic Brain Images: Bridging the Gap in Brain Mapping With Generative Adversarial Model [0.0]
This work investigates the use of Deep Convolutional Generative Adversarial Networks (DCGAN) for producing high-fidelity and realistic MRI image slices.
While the discriminator network discerns between created and real slices, the generator network learns to synthesise realistic MRI image slices.
The generator refines its capacity to generate slices that closely mimic real MRI data through an adversarial training approach.
arXiv Detail & Related papers (2024-04-11T05:06:51Z) - NeRF Solves Undersampled MRI Reconstruction [1.3597551064547502]
This article presents a novel undersampled magnetic resonance imaging (MRI) technique that leverages the concept of Neural Radiance Field (NeRF)
With radial undersampling, the corresponding imaging problem can be reformulated into an image modeling task from sparse-view rendered data.
A multi-layer perceptron, which is designed to output an image intensity from a spatial coordinate, learns the MR physics-driven rendering relation between given measurement data and desired image.
arXiv Detail & Related papers (2024-02-20T18:37:42Z) - Disentangled Latent Energy-Based Style Translation: An Image-Level Structural MRI Harmonization Framework [20.269574292365107]
We develop a novel framework for unpaired image-level MRI harmonization.
It consists of (a) site-invariant image generation ( SIG), (b) site-specific style translation (SST), and (c) site-specific MRI synthesis (SMS)
By disentangling image generation and style translation in latent space, the DLEST can achieve efficient style translation.
arXiv Detail & Related papers (2024-02-10T03:42:37Z) - Volumetric Reconstruction Resolves Off-Resonance Artifacts in Static and
Dynamic PROPELLER MRI [76.60362295758596]
Off-resonance artifacts in magnetic resonance imaging (MRI) are visual distortions that occur when the actual resonant frequencies of spins within the imaging volume differ from the expected frequencies used to encode spatial information.
We propose to resolve these artifacts by lifting the 2D MRI reconstruction problem to 3D, introducing an additional "spectral" dimension to model this off-resonance.
arXiv Detail & Related papers (2023-11-22T05:44:51Z) - Style transfer between Microscopy and Magnetic Resonance Imaging via
Generative Adversarial Network in small sample size settings [49.84018914962972]
Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising.
We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
arXiv Detail & Related papers (2023-10-16T13:58:53Z) - CMRxRecon: An open cardiac MRI dataset for the competition of
accelerated image reconstruction [62.61209705638161]
There has been growing interest in deep learning-based CMR imaging algorithms.
Deep learning methods require large training datasets.
This dataset includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects.
arXiv Detail & Related papers (2023-09-19T15:14:42Z) - Video4MRI: An Empirical Study on Brain Magnetic Resonance Image
Analytics with CNN-based Video Classification Frameworks [60.42012344842292]
3D CNN-based models dominate the field of magnetic resonance image (MRI) analytics.
In this paper, four datasets of Alzheimer's and Parkinson's disease recognition are utilized in experiments.
In terms of efficiency, the video framework performs better than 3D-CNN models by 5% - 11% with 50% - 66% less trainable parameters.
arXiv Detail & Related papers (2023-02-24T15:26:31Z) - Medical Diffusion -- Denoising Diffusion Probabilistic Models for 3D
Medical Image Generation [0.6486409713123691]
We show that diffusion probabilistic models can synthesize high quality medical imaging data.
We provide quantitative measurements of their performance through a reader study with two medical experts.
We demonstrate that synthetic images can be used in a self-supervised pre-training and improve the performance of breast segmentation models when data is scarce.
arXiv Detail & Related papers (2022-11-07T08:37:48Z) - Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks.
In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics.
Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
arXiv Detail & Related papers (2022-09-19T09:43:19Z) - Self-Supervised Multi-Modal Alignment for Whole Body Medical Imaging [70.52819168140113]
We use a dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans.
We introduce a multi-modal image-matching contrastive framework, that is able to learn to match different-modality scans of the same subject with high accuracy.
Without any adaption, we show that the correspondences learnt during this contrastive training step can be used to perform automatic cross-modal scan registration.
arXiv Detail & Related papers (2021-07-14T12:35:05Z) - Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance
Imaging -- Mini Review, Comparison and Perspectives [5.3148259096171175]
One drawback of MRI is its comparatively slow scanning and reconstruction compared to other image modalities.
Deep Neural Networks (DNNs) have been used in sparse MRI reconstruction models to recreate relatively high-quality images.
Generative Adversarial Networks (GAN) based methods are proposed to solve fast MRI with enhanced image perceptual quality.
arXiv Detail & Related papers (2021-05-04T23:59:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.