Multiscale Metamorphic VAE for 3D Brain MRI Synthesis
- URL: http://arxiv.org/abs/2301.03588v2
- Date: Wed, 11 Jan 2023 05:34:35 GMT
- Title: Multiscale Metamorphic VAE for 3D Brain MRI Synthesis
- Authors: Jaivardhan Kapoor, Jakob H. Macke, Christian F. Baumgartner
- Abstract summary: Generative modeling of 3D brain MRIs presents difficulties in achieving high visual fidelity while ensuring sufficient coverage of the data distribution.
In this work, we propose to address this challenge with composable, multiscale morphological transformations in a variational autoencoder framework.
We show substantial performance improvements in FID while retaining comparable, or superior, reconstruction quality compared to prior work based on VAEs and generative adversarial networks (GANs)
- Score: 5.060516201839319
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generative modeling of 3D brain MRIs presents difficulties in achieving high
visual fidelity while ensuring sufficient coverage of the data distribution. In
this work, we propose to address this challenge with composable, multiscale
morphological transformations in a variational autoencoder (VAE) framework.
These transformations are applied to a chosen reference brain image to generate
MRI volumes, equipping the model with strong anatomical inductive biases. We
structure the VAE latent space in a way such that the model covers the data
distribution sufficiently well. We show substantial performance improvements in
FID while retaining comparable, or superior, reconstruction quality compared to
prior work based on VAEs and generative adversarial networks (GANs).
Related papers
- 3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes [2.8498944632323755]
We propose a novel slice-based latent diffusion architecture to address the complexities of volumetric data generation.
This approach extends the joint distribution modeling of medical images and their associated masks, allowing a simultaneous generation of both under data-scarce regimes.
Our architecture can be conditioned by tumor characteristics, including size, shape, and relative position, thereby providing a diverse range of tumor variations.
arXiv Detail & Related papers (2024-06-08T09:53:45Z) - SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion
Classification Using 3D Multi-Phase Imaging [59.78761085714715]
This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification.
The proposed framework has been validated through comprehensive experiments on two clinical datasets.
To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
arXiv Detail & Related papers (2024-02-27T06:32:56Z) - A 3D generative model of pathological multi-modal MR images and
segmentations [3.4806591877889375]
We propose brainSPADE3D, a 3D generative model for brain MRI and associated segmentations.
The proposed joint imaging-segmentation generative model is shown to generate high-fidelity synthetic images and associated segmentations.
We demonstrate how the model can alleviate issues with segmentation model performance when unexpected pathologies are present in the data.
arXiv Detail & Related papers (2023-11-08T09:36:37Z) - Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D
Brain MRI Synthesis [35.45013834475523]
Cross-modality medical image synthesis is a critical topic and has the potential to facilitate numerous applications in the medical imaging field.
Most current medical image synthesis methods rely on generative adversarial networks and suffer from notorious mode collapse and unstable training.
We introduce a new paradigm for volumetric medical data synthesis by leveraging 2D backbones and present a diffusion-based framework, Make-A-Volume.
arXiv Detail & Related papers (2023-07-19T16:01:09Z) - On Sensitivity and Robustness of Normalization Schemes to Input
Distribution Shifts in Automatic MR Image Diagnosis [58.634791552376235]
Deep Learning (DL) models have achieved state-of-the-art performance in diagnosing multiple diseases using reconstructed images as input.
DL models are sensitive to varying artifacts as it leads to changes in the input data distribution between the training and testing phases.
We propose to use other normalization techniques, such as Group Normalization and Layer Normalization, to inject robustness into model performance against varying image artifacts.
arXiv Detail & Related papers (2023-06-23T03:09:03Z) - View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection.
First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan.
Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view.
Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
arXiv Detail & Related papers (2022-09-20T11:58:23Z) - Hierarchical Amortized Training for Memory-efficient High Resolution 3D
GAN [52.851990439671475]
We propose a novel end-to-end GAN architecture that can generate high-resolution 3D images.
We achieve this goal by using different configurations between training and inference.
Experiments on 3D thorax CT and brain MRI demonstrate that our approach outperforms state of the art in image generation.
arXiv Detail & Related papers (2020-08-05T02:33:04Z) - Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE [66.63629641650572]
We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices.
We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
arXiv Detail & Related papers (2020-07-09T13:23:15Z) - Scale-Space Autoencoders for Unsupervised Anomaly Segmentation in Brain
MRI [47.26574993639482]
We show improved anomaly segmentation performance and the general capability to obtain much more crisp reconstructions of input data at native resolution.
The modeling of the laplacian pyramid further enables the delineation and aggregation of lesions at multiple scales.
arXiv Detail & Related papers (2020-06-23T09:20:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.