XFMamba: Cross-Fusion Mamba for Multi-View Medical Image Classification

• URL: http://arxiv.org/abs/2503.02619v1
• Date: Tue, 04 Mar 2025 13:38:58 GMT
• Title: XFMamba: Cross-Fusion Mamba for Multi-View Medical Image Classification
• Authors: Xiaoyu Zheng, Xu Chen, Shaogang Gong, Xavier Griffin, Greg Slabaugh,
• Abstract summary: XFMamba is a pure Mamba-based cross-fusion architecture to address the challenge of multi-view medical image classification.<n>XFMamba introduces a novel two-stage fusion strategy, facilitating the learning of single-view features and their cross-view disparity.<n>Results show that it outperforms existing convolution-based and transformer-based multi-view methods.
• Score: 31.897467054280504
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Compared to single view medical image classification, using multiple views can significantly enhance predictive accuracy as it can account for the complementarity of each view while leveraging correlations between views. Existing multi-view approaches typically employ separate convolutional or transformer branches combined with simplistic feature fusion strategies. However, these approaches inadvertently disregard essential cross-view correlations, leading to suboptimal classification performance, and suffer from challenges with limited receptive field (CNNs) or quadratic computational complexity (transformers). Inspired by state space sequence models, we propose XFMamba, a pure Mamba-based cross-fusion architecture to address the challenge of multi-view medical image classification. XFMamba introduces a novel two-stage fusion strategy, facilitating the learning of single-view features and their cross-view disparity. This mechanism captures spatially long-range dependencies in each view while enhancing seamless information transfer between views. Results on three public datasets, MURA, CheXpert and DDSM, illustrate the effectiveness of our approach across diverse multi-view medical image classification tasks, showing that it outperforms existing convolution-based and transformer-based multi-view methods. Code is available at https://github.com/XZheng0427/XFMamba.

Related papers

• Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation [61.64052577026623]
  Real-world multi-view datasets are often heterogeneous and imperfect. We propose a novel robust MVL method (namely RML) with simultaneous representation fusion and alignment. In experiments, we employ it in unsupervised multi-view clustering, noise-label classification, and as a plug-and-play module for cross-modal hashing retrieval.
  arXiv  Detail & Related papers  (2025-03-06T07:01:08Z)
• MV-Swin-T: Mammogram Classification with Multi-view Swin Transformer [0.257133335028485]
  We propose an innovative multi-view network based on transformers to address challenges in mammographic image classification. Our approach introduces a novel shifted window-based dynamic attention block, facilitating the effective integration of multi-view information.
  arXiv  Detail & Related papers  (2024-02-26T04:41:04Z)
• Semi-Mamba-UNet: Pixel-Level Contrastive and Pixel-Level Cross-Supervised Visual Mamba-based UNet for Semi-Supervised Medical Image Segmentation [11.637738540262797]
  This study introduces Semi-Mamba-UNet, which integrates a purely visual Mamba-based encoder-decoder architecture with a conventional CNN-based UNet into a semi-supervised learning framework. This innovative SSL approach leverages both networks to generate pseudo-labels and cross-supervise one another at the pixel level simultaneously. We introduce a self-supervised pixel-level contrastive learning strategy that employs a pair of projectors to enhance the feature learning capabilities further.
  arXiv  Detail & Related papers  (2024-02-11T17:09:21Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network [67.97926983664676]
  We propose a cross-modal consistent multi-view medical report generation with a domain transfer network (C2M-DoT) C2M-DoT substantially outperforms state-of-the-art baselines in all metrics.
  arXiv  Detail & Related papers  (2023-10-09T02:31:36Z)
• DealMVC: Dual Contrastive Calibration for Multi-view Clustering [78.54355167448614]
  We propose a novel Dual contrastive calibration network for Multi-View Clustering (DealMVC) We first design a fusion mechanism to obtain a global cross-view feature. Then, a global contrastive calibration loss is proposed by aligning the view feature similarity graph and the high-confidence pseudo-label graph. During the training procedure, the interacted cross-view feature is jointly optimized at both local and global levels.
  arXiv  Detail & Related papers  (2023-08-17T14:14:28Z)
• CheXFusion: Effective Fusion of Multi-View Features using Transformers for Long-Tailed Chest X-Ray Classification [4.708378681950648]
  This paper introduces our solution to the ICCV CVAMD 2023 Shared Task on CXR-LT: Multi-Label Long-Tailed Classification on Chest X-Rays. Our approach introduces CheXFusion, a transformer-based fusion module incorporating multi-view images. Our solution achieves state-of-the-art results with 0.372 mAP in the MIMIC-CXR test set, securing 1st place in the competition.
  arXiv  Detail & Related papers  (2023-08-08T00:46:01Z)
• M$^3$Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition [80.21796574234287]
  M$3$Net is a matching-based framework for few-shot fine-grained (FS-FG) action recognition. It incorporates textitmulti-view encoding, textitmulti-view matching, and textitmulti-view fusion to facilitate embedding encoding, similarity matching, and decision making. Explainable visualizations and experimental results demonstrate the superiority of M$3$Net in capturing fine-grained action details.
  arXiv  Detail & Related papers  (2023-08-06T09:15:14Z)
• MORI-RAN: Multi-view Robust Representation Learning via Hybrid Contrastive Fusion [4.36488705757229]
  Multi-view representation learning is essential for many multi-view tasks, such as clustering and classification. We propose a hybrid contrastive fusion algorithm to extract robust view-common representation from unlabeled data. Experimental results demonstrated that the proposed method outperforms 12 competitive multi-view methods on four real-world datasets.
  arXiv  Detail & Related papers  (2022-08-26T09:58:37Z)
• Fast Multi-view Clustering via Ensembles: Towards Scalability, Superiority, and Simplicity [63.85428043085567]
  We propose a fast multi-view clustering via ensembles (FastMICE) approach. The concept of random view groups is presented to capture the versatile view-wise relationships. FastMICE has almost linear time and space complexity, and is free of dataset-specific tuning.
  arXiv  Detail & Related papers  (2022-03-22T09:51:24Z)
• TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers [8.139069987207494]
  We present TransFusion, a Transformer-based architecture to merge divergent multi-view imaging information using convolutional layers and powerful attention mechanisms. In particular, the Divergent Fusion Attention (DiFA) module is proposed for rich cross-view context modeling and semantic dependency mining.
  arXiv  Detail & Related papers  (2022-03-21T04:02:54Z)
• Deep Class-Specific Affinity-Guided Convolutional Network for Multimodal Unpaired Image Segmentation [7.021001169318551]
  Multi-modal medical image segmentation plays an essential role in clinical diagnosis. It remains challenging as the input modalities are often not well-aligned spatially. We propose an affinity-guided fully convolutional network for multimodal image segmentation.
  arXiv  Detail & Related papers  (2021-01-05T13:56:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.