CNN-ViT Fusion with Adaptive Attention Gate for Brain Tumor MRI Classification: A Hybrid Deep Learning Model

• URL: http://arxiv.org/abs/2604.23137v1
• Date: Sat, 25 Apr 2026 04:36:33 GMT
• Title: CNN-ViT Fusion with Adaptive Attention Gate for Brain Tumor MRI Classification: A Hybrid Deep Learning Model
• Abstract summary: We propose a new hybrid architecture that combines a SqueezeNet-style CNN branch with a MobileViT-style global transformer branch.<n>The gate learns dynamically per-sample, per-feature weights to weight the contribution of each branch, allowing context-sensitive merging of local and global representations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Early detection and classifying brain tumors using Magnetic Resonance Imaging (MRI) images is highly important but difficult to extract in medical images. Convolutional Neural Networks (CNNs) are good at capturing both local texture and spatial information whereas Vision Transformers (ViTs) are good at capturing long-range global dependencies. We propose a new hybrid architecture that combines a SqueezeNet-style CNN branch with a MobileViT-style global transformer branch, through an Adaptive Attention Gate mechanism, in this paper. The gate learns dynamically per-sample, per-feature weights to weight the contribution of each branch, allowing context-sensitive merging of local and global representations. The proposed model has a test accuracy of 97.60, a precision of 97.30, a recall of 97.50, an F1-score of 97.40, and a macro-average area under the curve (AUC) of 0.9946 with a trained and evaluated on the Brain Tumor MRI Dataset (Kaggle). These scores are higher than single CNN and ViT baselines, and current competitive fusion methods, showing that dynamic feature weighting is an effective way to classify medical images.

Related papers

• BrainFusionNet: a deep learning and XAI model to understand local, global, and sequential features of MRI images for improved brain tumour detection [6.183169673393073]
  We develop BrainFusionNet that combines Convolutional Neural Networks CNNs Vision Transformers ViT and Gated Recurrent Units GRUs to extract contextual and sequential features from MRI images for improved brain tumor classification.<n> explainable AI such as SHAP LIME and GradCAM are integrated to visualise and highlight image regions that contribute to BrainFusionNets decisionmaking process.
  arXiv  Detail & Related papers  (2026-06-17T04:20:46Z)
• Bridging Topology and Deep Representation Learning: A TDA-ViT Fusion Model for Four-Class Brain Tumor Classification [0.8374077003751697]
  We propose a fusion framework that combines Topological Data Analysis features with pretrained Vision Transformer representations for four-class brain tumor classification.<n>The proposed TDA-ViT fusion model achieves an accuracy of 99.10%, precision of 99.27%, recall of 99.15%, F1-score of 99.21%, and an AUC of 99.98%.
  arXiv  Detail & Related papers  (2026-05-30T23:28:15Z)
• Brain Tumor Classification in MRI Images: A Computationally Efficient Convolutional Neural Network [0.0]
  We propose a lightweight yet high-performing Convolutional Neural Network (CNN) for multi-class brain tumor classification.<n>The model was rigorously evaluated on two publicly accessible datasets from Figshare and Kaggle.<n>Our approach consistently demonstrated superior performance with reduced computational overhead.
  arXiv  Detail & Related papers  (2026-05-11T21:39:24Z)
• Adapting HFMCA to Graph Data: Self-Supervised Learning for Generalizable fMRI Representations [57.054499278843856]
  Functional magnetic resonance imaging (fMRI) analysis faces significant challenges due to limited dataset sizes and domain variability between studies.<n>Traditional self-supervised learning methods inspired by computer vision often rely on positive and negative sample pairs.<n>We propose adapting a recently developed Hierarchical Functional Maximal Correlation Algorithm (HFMCA) to graph-structured fMRI data.
  arXiv  Detail & Related papers  (2025-10-05T12:35:01Z)
• Brain Tumor Detection Through Diverse CNN Architectures in IoT Healthcare Industries: Fast R-CNN, U-Net, Transfer Learning-Based CNN, and Fully Connected CNN [0.42970700836450487]
  We classified glioma, meningioma, and pituitary tumors from MRI images using R-CNN, UNet, and transfer learning models.<n>The Fast R-CNN achieved the best results with 99% accuracy, 98.5% F-score, 99.5% Area Under the Curve (AUC), 99.4% recall, and 98.5% precision.<n>These findings underscore the robustness and reliability of AI models in handling diverse datasets.
  arXiv  Detail & Related papers  (2025-09-06T20:03:51Z)
• UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation [2.0544282587296974]
  Existing deep learning models face trade-offs between efficiency and accuracy.<n>We propose SAMA-UNet, a novel U-shaped architecture that introduces two key innovations.<n>Experiments on MRI, CT, and endoscopy datasets demonstrate that SAMA-UNet consistently outperforms CNN, Transformer, and Mamba-based methods.
  arXiv  Detail & Related papers  (2025-05-21T08:12:31Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [56.99710477905796]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Classification of Endoscopy and Video Capsule Images using CNN-Transformer Model [1.0994755279455526]
  This study proposes a hybrid model that combines the advantages of Transformers and Convolutional Neural Networks (CNNs) to enhance classification performance. For the GastroVision dataset, our proposed model demonstrates excellent performance with Precision, Recall, F1 score, Accuracy, and Matthews Correlation Coefficient (MCC) of 0.8320, 0.8386, 0.8324, 0.8386, and 0.8191, respectively.
  arXiv  Detail & Related papers  (2024-08-20T11:05:32Z)
• Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network [63.845552349914186]
  Capturing global contextual information plays a critical role in breast ultrasound (BUS) image classification. Vision Transformers have an improved capability of capturing global contextual information but may distort the local image patterns due to the tokenization operations. In this study, we proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN, designed to perform BUS tumor classification and segmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:19:32Z)
• Multiple Instance Ensembling For Paranasal Anomaly Classification In The Maxillary Sinus [46.1292414445895]
  Paranasal anomalies can present with a wide range of morphological features. Current approaches to paranasal anomaly classification are constrained to identifying a single anomaly at a time. We investigate the feasibility of using a 3D convolutional neural network (CNN) to classify healthy maxillary (MS) and MS with polyps or cysts.
  arXiv  Detail & Related papers  (2023-03-31T09:23:27Z)
• Hybrid Window Attention Based Transformer Architecture for Brain Tumor Segmentation [28.650980942429726]
  We propose a volumetric vision transformer that follows two windowing strategies in attention for extracting fine features. We trained and evaluated network architecture on the FeTS Challenge 2022 dataset. Our performance on the online validation dataset is as follows: Dice Similarity Score of 81.71%, 91.38% and 85.40%.
  arXiv  Detail & Related papers  (2022-09-16T03:55:48Z)
• Medulloblastoma Tumor Classification using Deep Transfer Learning with Multi-Scale EfficientNets [63.62764375279861]
  We propose an end-to-end MB tumor classification and explore transfer learning with various input sizes and matching network dimensions. Using a data set with 161 cases, we demonstrate that pre-trained EfficientNets with larger input resolutions lead to significant performance improvements.
  arXiv  Detail & Related papers  (2021-09-10T13:07:11Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.