Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks

• URL: http://arxiv.org/abs/2006.16741v1
• Date: Tue, 30 Jun 2020 12:58:41 GMT
• Title: Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks
• Authors: R. Robinson, Q. Dou, D.C. Castro, K. Kamnitsas, M. de Groot, R.M. Summers, D. Rueckert, B. Glocker
• Abstract summary: We use image-and-spatial transformer networks (ISTNs) to tackle domain shift in medical imaging data. We employ ISTNs for domain adaptation at the image-level which constrains transformations to explainable appearance and shape changes. For real-data validation, we construct two 3D brain MRI datasets from the Cam-CAN and UK Biobank studies.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate the use of image-and-spatial transformer networks (ISTNs) to tackle domain shift in multi-site medical imaging data. Commonly, domain adaptation (DA) is performed with little regard for explainability of the inter-domain transformation and is often conducted at the feature-level in the latent space. We employ ISTNs for DA at the image-level which constrains transformations to explainable appearance and shape changes. As proof-of-concept we demonstrate that ISTNs can be trained adversarially on a classification problem with simulated 2D data. For real-data validation, we construct two 3D brain MRI datasets from the Cam-CAN and UK Biobank studies to investigate domain shift due to acquisition and population differences. We show that age regression and sex classification models trained on ISTN output improve generalization when training on data from one and testing on the other site.

Related papers

• Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis [9.355513913682794]
  Current biomedical foundation models struggle to generalize as public 3D datasets are small. We propose a data engine that synthesizes highly variable training samples that enable generalization to new biomedical contexts. To then train a single 3D network for any voxel-level task, we develop a contrastive learning method that pretrains the network to be stable against nuisance imaging variation simulated by the data engine.
  arXiv  Detail & Related papers  (2024-11-04T18:40:46Z)
• M2OST: Many-to-one Regression for Predicting Spatial Transcriptomics from Digital Pathology Images [16.19308597273405]
  We propose M2OST, a many-to-one regression Transformer that can accommodate the hierarchical structure of pathology images. Unlike traditional models that are trained with one-to-one image-label pairs, M2OST uses multiple images from different levels of the digital pathology image to jointly predict the gene expressions in their common corresponding spot. M2OST can achieve state-of-the-art performance with fewer parameters and floating-point operations (FLOPs)
  arXiv  Detail & Related papers  (2024-09-23T15:06:37Z)
• Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers [48.74331852418905]
  Direct image-to-graph transformation is a challenging task that involves solving object detection and relationship prediction in a single model. Due to this task's complexity, large training datasets are rare in many domains, making the training of deep-learning methods challenging. We introduce a set of methods enabling cross-domain and cross-dimension learning for image-to-graph transformers.
  arXiv  Detail & Related papers  (2024-03-11T10:48:56Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• IGUANe: a 3D generalizable CycleGAN for multicenter harmonization of brain MR images [0.0]
  Deep learning methods for image translation have emerged as a solution for harmonizing MR images across sites. In this study, we introduce IGUANe, an original 3D model that leverages the strengths of domain translation. The framework based on domain pairs enables the implementation of sampling strategies that prevent confusion between site-related and biological variabilities.
  arXiv  Detail & Related papers  (2024-02-05T17:38:49Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation [61.01704175938995]
  In this paper, we propose a novel Bidirectional Global-to-Local (BiGL) adaptation framework under a UDA scheme. Specifically, a bidirectional image synthesis and segmentation module is proposed to segment the brain tumor. The proposed method outperforms several state-of-the-art unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2021-05-17T10:11:45Z)
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation [9.659642285903418]
  Cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups.
  arXiv  Detail & Related papers  (2021-03-05T16:22:31Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.