Brain tumor segmentation with missing modalities via latent multi-source correlation representation

• URL: http://arxiv.org/abs/2003.08870v5
• Date: Tue, 20 Apr 2021 12:27:22 GMT
• Title: Brain tumor segmentation with missing modalities via latent multi-source correlation representation
• Authors: Tongxue Zhou, St\'ephane Canu, Pierre Vera, Su Ruan
• Abstract summary: A novel correlation representation block is proposed to specially discover the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modalities. We evaluate our model on BraTS 2018 datasets, it outperforms the current state-of-the-art method and produces robust results when one or more modalities are missing.
• Score: 6.060020806741279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal MR images can provide complementary information for accurate brain tumor segmentation. However, it's common to have missing imaging modalities in clinical practice. Since there exists a strong correlation between multi modalities, a novel correlation representation block is proposed to specially discover the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modalities. The model parameter estimation module first maps the individual representation produced by each encoder to obtain independent parameters, then, under these parameters, the correlation expression module transforms all the individual representations to form a latent multi-source correlation representation. Finally, the correlation representations across modalities are fused via the attention mechanism into a shared representation to emphasize the most important features for segmentation. We evaluate our model on BraTS 2018 datasets, it outperforms the current state-of-the-art method and produces robust results when one or more modalities are missing.

Related papers

• Representation Surgery for Multi-Task Model Merging [57.63643005215592]
  Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training. By visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias.
  arXiv  Detail & Related papers  (2024-02-05T03:39:39Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• I2SRM: Intra- and Inter-Sample Relationship Modeling for Multimodal Information Extraction [10.684005956288347]
  We present the Intra- and Inter-Sample Relationship Modeling (I2SRM) method for this task. Our proposed method achieves competitive results, 77.12% F1-score on Twitter-2015, 88.40% F1-score on Twitter-2017, and 84.12% F1-score on MNRE.
  arXiv  Detail & Related papers  (2023-10-10T05:50:25Z)
• Exploiting Partial Common Information Microstructure for Multi-Modal Brain Tumor Segmentation [11.583406152227637]
  Learning with multiple modalities is crucial for automated brain tumor segmentation from magnetic resonance imaging data. Existing approaches are oblivious to partial common information shared by subsets of the modalities. In this paper, we show that identifying such partial common information can significantly boost the discriminative power of image segmentation models.
  arXiv  Detail & Related papers  (2023-02-06T01:28:52Z)
• A Tri-attention Fusion Guided Multi-modal Segmentation Network [2.867517731896504]
  We propose a multi-modality segmentation network guided by a novel tri-attention fusion. Our network includes N model-independent encoding paths with N image sources, a tri-attention fusion block, a dual-attention fusion block, and a decoding path. Our experiment results tested on BraTS 2018 dataset for brain tumor segmentation demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2021-11-02T14:36:53Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)
• Learning Multimodal VAEs through Mutual Supervision [72.77685889312889]
  MEME combines information between modalities implicitly through mutual supervision. We demonstrate that MEME outperforms baselines on standard metrics across both partial and complete observation schemes.
  arXiv  Detail & Related papers  (2021-06-23T17:54:35Z)
• Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities [2.867517731896504]
  Accurately segmenting brain tumor from MR images is the key to clinical diagnostics and treatment planning. It's common to miss some imaging modalities in clinical practice. We present a novel brain tumor segmentation algorithm with missing modalities.
  arXiv  Detail & Related papers  (2021-04-13T14:21:09Z)
• 3D Medical Multi-modal Segmentation Network Guided by Multi-source Correlation Constraint [2.867517731896504]
  We propose a multi-modality segmentation network with a correlation constraint. Our experiment results tested on BraTS-2018 dataset for brain tumor segmentation demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2021-02-05T11:23:12Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)
• Modeling Shared Responses in Neuroimaging Studies through MultiView ICA [94.31804763196116]
  Group studies involving large cohorts of subjects are important to draw general conclusions about brain functional organization. We propose a novel MultiView Independent Component Analysis model for group studies, where data from each subject are modeled as a linear combination of shared independent sources plus noise. We demonstrate the usefulness of our approach first on fMRI data, where our model demonstrates improved sensitivity in identifying common sources among subjects.
  arXiv  Detail & Related papers  (2020-06-11T17:29:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.