Image-level supervision and self-training for transformer-based
cross-modality tumor segmentation
- URL: http://arxiv.org/abs/2309.09246v1
- Date: Sun, 17 Sep 2023 11:50:12 GMT
- Title: Image-level supervision and self-training for transformer-based
cross-modality tumor segmentation
- Authors: Malo de Boisredon and Eugene Vorontsov and William Trung Le and Samuel
Kadoury
- Abstract summary: We propose a new semi-supervised training strategy called MoDATTS.
MoDATTS is designed for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets.
We report that 99% and 100% of this maximum performance can be attained if 20% and 50% of the target data is annotated.
- Score: 2.29206349318258
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep neural networks are commonly used for automated medical image
segmentation, but models will frequently struggle to generalize well across
different imaging modalities. This issue is particularly problematic due to the
limited availability of annotated data, making it difficult to deploy these
models on a larger scale. To overcome these challenges, we propose a new
semi-supervised training strategy called MoDATTS. Our approach is designed for
accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets. An
image-to-image translation strategy between imaging modalities is used to
produce annotated pseudo-target volumes and improve generalization to the
unannotated target modality. We also use powerful vision transformer
architectures and introduce an iterative self-training procedure to further
close the domain gap between modalities. MoDATTS additionally allows the
possibility to extend the training to unannotated target data by exploiting
image-level labels with an unsupervised objective that encourages the model to
perform 3D diseased-to-healthy translation by disentangling tumors from the
background. The proposed model achieves superior performance compared to other
methods from participating teams in the CrossMoDA 2022 challenge, as evidenced
by its reported top Dice score of 0.87+/-0.04 for the VS segmentation. MoDATTS
also yields consistent improvements in Dice scores over baselines on a
cross-modality brain tumor segmentation task composed of four different
contrasts from the BraTS 2020 challenge dataset, where 95% of a target
supervised model performance is reached. We report that 99% and 100% of this
maximum performance can be attained if 20% and 50% of the target data is
additionally annotated, which further demonstrates that MoDATTS can be
leveraged to reduce the annotation burden.
Related papers
- Discriminative Hamiltonian Variational Autoencoder for Accurate Tumor Segmentation in Data-Scarce Regimes [2.8498944632323755]
We propose an end-to-end hybrid architecture for medical image segmentation.
We use Hamiltonian Variational Autoencoders (HVAE) and a discriminative regularization to improve the quality of generated images.
Our architecture operates on a slice-by-slice basis to segment 3D volumes, capitilizing on the richly augmented dataset.
arXiv Detail & Related papers (2024-06-17T15:42:08Z) - 3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes [2.8498944632323755]
We propose a novel slice-based latent diffusion architecture to address the complexities of volumetric data generation.
This approach extends the joint distribution modeling of medical images and their associated masks, allowing a simultaneous generation of both under data-scarce regimes.
Our architecture can be conditioned by tumor characteristics, including size, shape, and relative position, thereby providing a diverse range of tumor variations.
arXiv Detail & Related papers (2024-06-08T09:53:45Z) - Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement
Learning [53.00683059396803]
Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images.
We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy.
Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
arXiv Detail & Related papers (2023-10-06T10:40:46Z) - Diffusion Models for Counterfactual Generation and Anomaly Detection in
Brain Images [59.85702949046042]
We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map.
We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process.
We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications.
arXiv Detail & Related papers (2023-08-03T21:56:50Z) - Cross-modal tumor segmentation using generative blending augmentation and self training [1.6440045168835438]
We propose a cross-modal segmentation method based on conventional image synthesis boosted by a new data augmentation technique.
Generative Blending Augmentation (GBA) learns representative generative features from a single training image to realistically diversify tumor appearances.
The proposed solution ranked first for vestibular schwannoma (VS) segmentation during the validation and test phases of the MICCAI CrossMoDA 2022 challenge.
arXiv Detail & Related papers (2023-04-04T11:01:46Z) - AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context
Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
We present a novel concept of shared-context processing for whole slide histopathology images.
AMIGO uses the celluar graph within the tissue to provide a single representation for a patient.
We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
arXiv Detail & Related papers (2023-03-01T23:37:45Z) - M-GenSeg: Domain Adaptation For Target Modality Tumor Segmentation With
Annotation-Efficient Supervision [4.023899199756184]
M-GenSeg is a new semi-supervised generative training strategy for cross-modality tumor segmentation.
We evaluate the performance on a brain tumor segmentation dataset composed of four different contrast sequences.
Unlike the prior art, M-GenSeg also introduces the ability to train with a partially annotated source modality.
arXiv Detail & Related papers (2022-12-14T15:19:06Z) - Stacking Ensemble Learning in Deep Domain Adaptation for Ophthalmic
Image Classification [61.656149405657246]
Domain adaptation is effective in image classification tasks where obtaining sufficient label data is challenging.
We propose a novel method, named SELDA, for stacking ensemble learning via extending three domain adaptation methods.
The experimental results using Age-Related Eye Disease Study (AREDS) benchmark ophthalmic dataset demonstrate the effectiveness of the proposed model.
arXiv Detail & Related papers (2022-09-27T14:19:00Z) - Cross-Modality Brain Tumor Segmentation via Bidirectional
Global-to-Local Unsupervised Domain Adaptation [61.01704175938995]
In this paper, we propose a novel Bidirectional Global-to-Local (BiGL) adaptation framework under a UDA scheme.
Specifically, a bidirectional image synthesis and segmentation module is proposed to segment the brain tumor.
The proposed method outperforms several state-of-the-art unsupervised domain adaptation methods by a large margin.
arXiv Detail & Related papers (2021-05-17T10:11:45Z) - Realistic Adversarial Data Augmentation for MR Image Segmentation [17.951034264146138]
We propose an adversarial data augmentation method for training neural networks for medical image segmentation.
Our model generates plausible and realistic signal corruptions, which models the intensity inhomogeneities caused by a common type of artefacts in MR imaging: bias field.
We show that such an approach can improve the ability generalization and robustness of models as well as provide significant improvements in low-data scenarios.
arXiv Detail & Related papers (2020-06-23T20:43:18Z) - Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement
and Gated Fusion [71.87627318863612]
We propose a novel multimodal segmentation framework which is robust to the absence of imaging modalities.
Our network uses feature disentanglement to decompose the input modalities into the modality-specific appearance code.
We validate our method on the important yet challenging multimodal brain tumor segmentation task with the BRATS challenge dataset.
arXiv Detail & Related papers (2020-02-22T14:32:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.