Multimodal Brain Tumour Classification Using Feature Fusion

• URL: http://arxiv.org/abs/2606.11107v2
• Date: Wed, 10 Jun 2026 10:48:59 GMT
• Title: Multimodal Brain Tumour Classification Using Feature Fusion
• Abstract summary: Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement.<n>We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features to classify brain tumors into glioma, meningioma, pituitary, and notumor.<n>All multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.
• Score: 0.9021888125320424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement. However, most deep learning models rely on MRI/CT images alone, failing to replicate the clinicians multimodal reasoning. We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features (intensity, texture, shape, and boundary descriptors) to classify brain tumors into glioma, meningioma, pituitary, and no-tumor. A pre-trained CNN backbone encodes the image stream, whereas a dedicated MLP encodes the radiomic stream. Both streams are fused via concatenation, gated, or bidirectional cross-modal attention strategies. Across nine experimental runs on a balanced 7,200 image dataset, all multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.

Related papers

• CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing [3.6380774806458214]
  We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing.<n>The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR)<n>All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks.
  arXiv  Detail & Related papers  (2026-03-04T01:06:50Z)
• brat: Aligned Multi-View Embeddings for Brain MRI Analysis [36.795218160666266]
  brat is a multi-view representation learning framework for brain magnetic resonance imaging (MRI) trained on MRIs paired with clinical reports.<n>Brain MRIs present unique challenges due to the presence of numerous, highly varied, and often subtle abnormalities that are localized to a few slices within a 3D volume.
  arXiv  Detail & Related papers  (2025-12-21T10:37:31Z)
• Multi-modal Vision Pre-training for Medical Image Analysis [11.569448567735435]
  Self-supervised learning has greatly facilitated medical image analysis by suppressing the training data requirement for real-world applications.<n>We conduct a novel multi-modal image pre-training with three proxy tasks to facilitate the learning of cross-modality representations and correlations.<n>Our method is reported in comparison to state-of-the-art pre-training methods, with Dice Score improvement of 0.28%-14.47% across six segmentation benchmarks and a consistent accuracy boost of 0.65%-18.07% in four individual image classification tasks.
  arXiv  Detail & Related papers  (2024-10-14T15:12:16Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings [45.62331048595689]
  Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising.<n>We tested a method for generating microscopic histological images from MRI scans of the human corpus callosum using conditional generative adversarial network (cGAN) architecture.
  arXiv  Detail & Related papers  (2023-10-16T13:58:53Z)
• CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans [45.83431075462771]
  Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice. Most medical AI systems are built to focus on single organs with a narrow list of a few diseases. CancerUniT is a query-based Mask Transformer model with the output of multi-tumor prediction.
  arXiv  Detail & Related papers  (2023-01-28T20:09:34Z)
• CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation [37.39921484146194]
  Brain tumor segmentation in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes. With the great success of the ten-year BraTS challenges, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects. We propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS.
  arXiv  Detail & Related papers  (2022-07-15T09:35:29Z)
• mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation [38.22852533584288]
  We propose a novel Medical Transformer (mmFormer) for incomplete multimodal learning with three main components. The proposed mmFormer outperforms the state-of-the-art methods for incomplete multimodal brain tumor segmentation on almost all subsets of incomplete modalities.
  arXiv  Detail & Related papers  (2022-06-06T08:41:56Z)
• Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion [3.9562534927482704]
  We propose an end-to-end framework named Modality-Level Attention Fusion Network (MAF-Net) Our proposed MAF-Net is found to yield superior T1ce synthesis performance and accurate brain tumor segmentation.
  arXiv  Detail & Related papers  (2022-03-09T09:08:48Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)
• Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities [2.867517731896504]
  Accurately segmenting brain tumor from MR images is the key to clinical diagnostics and treatment planning. It's common to miss some imaging modalities in clinical practice. We present a novel brain tumor segmentation algorithm with missing modalities.
  arXiv  Detail & Related papers  (2021-04-13T14:21:09Z)
• Soft Tissue Sarcoma Co-Segmentation in Combined MRI and PET/CT Data [2.2515303891664358]
  Tumor segmentation in multimodal medical images has seen a growing trend towards deep learning based methods. We propose a simultaneous co-segmentation method, which enables multimodal feature learning through modality-specific encoder and decoder branches. We demonstrate the effectiveness of our approach on public soft tissue sarcoma data, which comprises MRI (T1 and T2 sequence) and PET/CT scans.
  arXiv  Detail & Related papers  (2020-08-28T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.