Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks

• URL: http://arxiv.org/abs/2006.16741v1
• Date: Tue, 30 Jun 2020 12:58:41 GMT
• Title: Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks
• Authors: R. Robinson, Q. Dou, D.C. Castro, K. Kamnitsas, M. de Groot, R.M. Summers, D. Rueckert, B. Glocker
• Abstract summary: We use image-and-spatial transformer networks (ISTNs) to tackle domain shift in medical imaging data. We employ ISTNs for domain adaptation at the image-level which constrains transformations to explainable appearance and shape changes. For real-data validation, we construct two 3D brain MRI datasets from the Cam-CAN and UK Biobank studies.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate the use of image-and-spatial transformer networks (ISTNs) to tackle domain shift in multi-site medical imaging data. Commonly, domain adaptation (DA) is performed with little regard for explainability of the inter-domain transformation and is often conducted at the feature-level in the latent space. We employ ISTNs for DA at the image-level which constrains transformations to explainable appearance and shape changes. As proof-of-concept we demonstrate that ISTNs can be trained adversarially on a classification problem with simulated 2D data. For real-data validation, we construct two 3D brain MRI datasets from the Cam-CAN and UK Biobank studies to investigate domain shift due to acquisition and population differences. We show that age regression and sex classification models trained on ISTN output improve generalization when training on data from one and testing on the other site.

Related papers

• Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis [9.355513913682794]
  Current biomedical foundation models struggle to generalize as public 3D datasets are small. We propose a data engine that synthesizes highly variable training samples that enable generalization to new biomedical contexts. To then train a single 3D network for any voxel-level task, we develop a contrastive learning method that pretrains the network to be stable against nuisance imaging variation simulated by the data engine.
  arXiv  Detail & Related papers  (2024-11-04T18:40:46Z)
• M2OST: Many-to-one Regression for Predicting Spatial Transcriptomics from Digital Pathology Images [16.19308597273405]
  We propose M2OST, a many-to-one regression Transformer that can accommodate the hierarchical structure of pathology images. Unlike traditional models that are trained with one-to-one image-label pairs, M2OST uses multiple images from different levels of the digital pathology image to jointly predict the gene expressions in their common corresponding spot. M2OST can achieve state-of-the-art performance with fewer parameters and floating-point operations (FLOPs)
  arXiv  Detail & Related papers  (2024-09-23T15:06:37Z)
• Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers [50.576354045312115]
  Direct image-to-graph transformation is a challenging task that solves object detection and relationship prediction in a single model. We introduce a set of methods enabling cross-domain and cross-dimension transfer learning for image-to-graph transformers. We demonstrate our method's utility in cross-domain and cross-dimension experiments, where we pretrain our models on 2D satellite images before applying them to vastly different target domains in 2D and 3D.
  arXiv  Detail & Related papers  (2024-03-11T10:48:56Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• IGUANe: a 3D generalizable CycleGAN for multicenter harmonization of brain MR images [0.0]
  Deep learning methods for image translation have emerged as a solution for harmonizing MR images across sites. In this study, we introduce IGUANe, an original 3D model that leverages the strengths of domain translation. The framework based on domain pairs enables the implementation of sampling strategies that prevent confusion between site-related and biological variabilities.
  arXiv  Detail & Related papers  (2024-02-05T17:38:49Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation [61.01704175938995]
  In this paper, we propose a novel Bidirectional Global-to-Local (BiGL) adaptation framework under a UDA scheme. Specifically, a bidirectional image synthesis and segmentation module is proposed to segment the brain tumor. The proposed method outperforms several state-of-the-art unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2021-05-17T10:11:45Z)
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation [9.659642285903418]
  Cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups.
  arXiv  Detail & Related papers  (2021-03-05T16:22:31Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)
• Medical Image Harmonization Using Deep Learning Based Canonical Mapping: Toward Robust and Generalizable Learning in Imaging [4.396671464565882]
  We propose a new paradigm in which data from a diverse range of acquisition conditions are "harmonized" to a common reference domain. We test this approach on two example problems, namely MRI-based brain age prediction and classification of schizophrenia.
  arXiv  Detail & Related papers  (2020-10-11T22:01:37Z)
• Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation [17.504340316130023]
  Our method is inspired by the fact that the spatial relationship between internal structures in medical images is relatively fixed. We formulate the spatial relationship by solving a jigsaw puzzle task, recovering a CT scan from its shuffled patches, and jointly train it with the organ segmentation task. To guarantee the transferability of the learned spatial relationship to multiple domains, we introduce two schemes: 1) Employing a super-resolution network also jointly trained with the segmentation model to standardize medical images from different domain to a certain spatial resolution; 2) Adapting the spatial relationship for a test image by test-time jigsaw puzzle training.
  arXiv  Detail & Related papers  (2020-05-18T22:44:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.