Cross-Attention Multimodal Fusion for Breast Cancer Diagnosis: Integrating Mammography and Clinical Data with Explainability

• URL: http://arxiv.org/abs/2508.16000v1
• Date: Thu, 21 Aug 2025 23:23:06 GMT
• Title: Cross-Attention Multimodal Fusion for Breast Cancer Diagnosis: Integrating Mammography and Clinical Data with Explainability
• Authors: Muhaisin Tiyumba Nantogmah, Abdul-Barik Alhassan, Salamudeen Alhassan,
• Abstract summary: Current computer-aided systems only use characteristics from mammograms.<n>Will clinical features greatly enhance the categorisation of breast lesions?<n>In what ways may explainable AI approaches improve the interpretability and reliability of models used to diagnose breast cancer?
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A precise assessment of the risk of breast lesions can greatly lower it and assist physicians in choosing the best course of action. To categorise breast lesions, the majority of current computer-aided systems only use characteristics from mammograms. Although this method is practical, it does not completely utilise clinical reports' valuable information to attain the best results. When compared to utilising mammography alone, will clinical features greatly enhance the categorisation of breast lesions? How may clinical features and mammograms be combined most effectively? In what ways may explainable AI approaches improve the interpretability and reliability of models used to diagnose breast cancer? To answer these basic problems, a comprehensive investigation is desperately needed. In order to integrate mammography and categorical clinical characteristics, this study examines a number of multimodal deep networks grounded on feature concatenation, co-attention, and cross-attention. The model achieved an AUC-ROC of 0.98, accuracy of 0.96, F1-score of 0.94, precision of 0.92, and recall of 0.95 when tested on publicly accessible datasets (TCGA and CBIS-DDSM).

Related papers

• An Explainable Hybrid AI Framework for Enhanced Tuberculosis and Symptom Detection [55.35661671061754]
  Tuberculosis remains a critical global health issue, particularly in resource-limited and remote areas.<n>We propose a framework which enhances disease and symptom detection on chest X-rays by integrating two supervised heads and a self-supervised head.<n>Our model achieves an accuracy of 98.85% for distinguishing between COVID-19, tuberculosis, and normal cases, and a macro-F1 score of 90.09% for multilabel symptom detection.
  arXiv  Detail & Related papers  (2025-10-21T17:18:55Z)
• Bridging Accuracy and Interpretability: Deep Learning with XAI for Breast Cancer Detection [0.0]
  We present an interpretable deep learning framework for the early detection of breast cancer using quantitative features extracted from digitized fine needle aspirate (FNA) images of breast masses.<n>Our deep neural network, using ReLU activations, the Adam visualizations, and a binary cross-entropy loss, delivers state-of-the-art classification performance.
  arXiv  Detail & Related papers  (2025-10-18T07:47:26Z)
• A Versatile Foundation Model for AI-enabled Mammogram Interpretation [36.48796962019666]
  VersaMammo is a versatile foundation model for mammograms designed to overcome limitations.<n>First, a teacher model is trained via self-supervised learning to extract transferable features from unlabeled mammograms.<n>Then, supervised learning combined with knowledge distillation transfers both features and clinical knowledge into VersaMammo.
  arXiv  Detail & Related papers  (2025-09-24T16:04:39Z)
• Prostate Cancer Classification Using Multimodal Feature Fusion and Explainable AI [2.656041527404895]
  We propose an explainable AI system combining BERT (for textual clinical notes) and Random Forest (for numerical lab data)<n>Our work demonstrates that a simple yet interpretable BERT+RF pipeline delivers clinically significant improvements.
  arXiv  Detail & Related papers  (2025-07-28T11:07:17Z)
• Adversarially Robust Feature Learning for Breast Cancer Diagnosis [1.9060744600841881]
  Adversarial data can lead to malfunction of deep learning applications. We propose a novel adversarially robust feature learning (ARFL) method for a real-world application of breast cancer diagnosis.
  arXiv  Detail & Related papers  (2024-02-13T20:02:34Z)
• CIMIL-CRC: a clinically-informed multiple instance learning framework for patient-level colorectal cancer molecular subtypes classification from H\&E stained images [42.771819949806655]
  We introduce CIMIL-CRC', a framework that solves the MSI/MSS MIL problem by efficiently combining a pre-trained feature extraction model with principal component analysis (PCA) to aggregate information from all patches. We assessed our CIMIL-CRC method using the average area under the curve (AUC) from a 5-fold cross-validation experimental setup for model development on the TCGA-CRC-DX cohort.
  arXiv  Detail & Related papers  (2024-01-29T12:56:11Z)
• Domain Transfer Through Image-to-Image Translation for Uncertainty-Aware Prostate Cancer Classification [42.75911994044675]
  We present a novel approach for unpaired image-to-image translation of prostate MRIs and an uncertainty-aware training approach for classifying clinically significant PCa. Our approach involves a novel pipeline for translating unpaired 3.0T multi-parametric prostate MRIs to 1.5T, thereby augmenting the available training data. Our experiments demonstrate that the proposed method significantly improves the Area Under ROC Curve (AUC) by over 20% compared to the previous work.
  arXiv  Detail & Related papers  (2023-07-02T05:26:54Z)
• Deep-Learning Tool for Early Identifying Non-Traumatic Intracranial Hemorrhage Etiology based on CT Scan [40.51754649947294]
  The deep learning model was developed with 1868 eligible NCCT scans with non-traumatic ICH collected between January 2011 and April 2018. The model's diagnostic performance was compared with clinicians's performance. The clinicians achieve significant improvements in the sensitivity, specificity, and accuracy of diagnoses of certain hemorrhage etiologies with proposed system augmentation.
  arXiv  Detail & Related papers  (2023-02-02T08:45:17Z)
• High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection [48.88813637974911]
  Computer-aided detection systems based on deep learning have shown good performance in breast cancer detection. High-density breasts show poorer detection performance since dense tissues can mask or even simulate masses. This study aims to improve the mass detection performance in high-density breasts using synthetic high-density full-field digital mammograms.
  arXiv  Detail & Related papers  (2022-09-20T15:57:12Z)
• Improving the diagnosis of breast cancer based on biophysical ultrasound features utilizing machine learning [0.0]
  We propose a biophysical feature based machine learning method for breast cancer detection. The overall accuracy for the most common types and sizes of breast lesions in our study exceeded 98.0% for classification and 0.98 for an area under the receiver operating characteristic curve.
  arXiv  Detail & Related papers  (2022-07-13T23:53:09Z)
• Translating automated brain tumour phenotyping to clinical neuroimaging [0.4199844472131921]
  We use state-of-the-art methods to quantify the comparative fidelity of automated tumour segmentation models. Deep learning segmentation models characterize tumours well when missing data and can even detect enhancing tissue without the use of contrast.
  arXiv  Detail & Related papers  (2022-06-13T12:58:54Z)
• EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer [58.720142291102135]
  We propose an efficient and light-weighted learning architecture to classify and segment breast tumors simultaneously. We incorporate a segmentation task into a tumor classification network, which makes the backbone network learn representations focused on tumor regions. The accuracy, sensitivity, and specificity of tumor classification is 88.6%, 94.1%, and 85.3%, respectively.
  arXiv  Detail & Related papers  (2022-01-13T05:24:40Z)
• Multimodal Breast Lesion Classification Using Cross-Attention Deep Networks [0.08635315042809139]
  Most computer-aided systems rely solely on mammogram features to classify breast lesions. This paper investigates several multimodal deep networks based on feature concatenation, cross-attention, and co-attention to combine mammograms and categorical clinical variables.
  arXiv  Detail & Related papers  (2021-08-21T23:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.