Multimodal Breast Lesion Classification Using Cross-Attention Deep Networks

• URL: http://arxiv.org/abs/2108.09591v1
• Date: Sat, 21 Aug 2021 23:01:31 GMT
• Title: Multimodal Breast Lesion Classification Using Cross-Attention Deep Networks
• Authors: Hung Q. Vo, Pengyu Yuan, Tiancheng He, Stephen T.C. Wong, and Hien V. Nguyen
• Abstract summary: Most computer-aided systems rely solely on mammogram features to classify breast lesions. This paper investigates several multimodal deep networks based on feature concatenation, cross-attention, and co-attention to combine mammograms and categorical clinical variables.
• Score: 0.08635315042809139
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate breast lesion risk estimation can significantly reduce unnecessary biopsies and help doctors decide optimal treatment plans. Most existing computer-aided systems rely solely on mammogram features to classify breast lesions. While this approach is convenient, it does not fully exploit useful information in clinical reports to achieve the optimal performance. Would clinical features significantly improve breast lesion classification compared to using mammograms alone? How to handle missing clinical information caused by variation in medical practice? What is the best way to combine mammograms and clinical features? There is a compelling need for a systematic study to address these fundamental questions. This paper investigates several multimodal deep networks based on feature concatenation, cross-attention, and co-attention to combine mammograms and categorical clinical variables. We show that the proposed architectures significantly increase the lesion classification performance (average area under ROC curves from 0.89 to 0.94). We also evaluate the model when clinical variables are missing.

Related papers

• Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data [76.89269238957593]
  Coronary artery stenosis is a leading cause of cardiovascular disease, diagnosed by analyzing the coronary arteries from multiple angiography views.<n>We propose SegmentMIL, a transformer-based multi-view multiple-instance learning framework for patient-level stenosis classification.
  arXiv  Detail & Related papers  (2026-02-02T13:07:52Z)
• Cross-Attention Multimodal Fusion for Breast Cancer Diagnosis: Integrating Mammography and Clinical Data with Explainability [0.0]
  Current computer-aided systems only use characteristics from mammograms.<n>Will clinical features greatly enhance the categorisation of breast lesions?<n>In what ways may explainable AI approaches improve the interpretability and reliability of models used to diagnose breast cancer?
  arXiv  Detail & Related papers  (2025-08-21T23:23:06Z)
• Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography [39.58317527488534]
  This study compares multimodal and CNN-based methods for automated classification using the BI-RADS system.<n>Zero-shot classification achieved modest performance, while the fine-tuned ConvNeXt model outperformed the BioMedCLIP linear probe.<n>These findings suggest that despite the promise of multimodal learning, CNN-based models with end-to-end fine-tuning provide stronger performance for specialized medical imaging.
  arXiv  Detail & Related papers  (2025-06-16T20:14:37Z)
• An Artificial Intelligence Model for Early Stage Breast Cancer Detection from Biopsy Images [0.0]
  This paper presents an artificial intelligence enabled tool designed to aid in the identification of breast cancer types.<n>The proposed model utilizes a convolutional neural network (CNN) architecture to distinguish between benign and malignant tissues.<n> Experimental results on such datasets demonstrate the model's effectiveness, outperforming several existing solutions in terms of accuracy, precision, recall, and F1-score.
  arXiv  Detail & Related papers  (2025-05-24T09:11:50Z)
• Revisiting Invariant Learning for Out-of-Domain Generalization on Multi-Site Mammogram Datasets [8.080495390226115]
  This paper reassesses the application of invariant learning for breast cancer risk estimation based on mammograms. Evaluation metrics include accuracy, average precision, and area under the curve. This research examines the advantages, limitations, and challenges of invariant learning for mammogram classification.
  arXiv  Detail & Related papers  (2025-03-09T20:28:04Z)
• Multi-task Explainable Skin Lesion Classification [54.76511683427566]
  We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
  arXiv  Detail & Related papers  (2023-10-11T05:49:47Z)
• Breast Cancer Classification Based on Histopathological Images Using a Deep Learning Capsule Network [0.0]
  This study aims to classify different types of breast cancer using histological images (HIs) We present an enhanced capsule network that extracts multi-scale features using the Res2Net block and four additional convolutional layers. As a result, the new method outperforms the old ones since it automatically learns the best possible features.
  arXiv  Detail & Related papers  (2022-08-01T03:45:36Z)
• Application of Transfer Learning and Ensemble Learning in Image-level Classification for Breast Histopathology [9.037868656840736]
  In Computer-Aided Diagnosis (CAD), traditional classification models mostly use a single network to extract features. This paper proposes a deep ensemble model based on image-level labels for the binary classification of benign and malignant lesions. Result: In the ensemble network model with accuracy as the weight, the image-level binary classification achieves an accuracy of $98.90%$.
  arXiv  Detail & Related papers  (2022-04-18T13:31:53Z)
• EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer [58.720142291102135]
  We propose an efficient and light-weighted learning architecture to classify and segment breast tumors simultaneously. We incorporate a segmentation task into a tumor classification network, which makes the backbone network learn representations focused on tumor regions. The accuracy, sensitivity, and specificity of tumor classification is 88.6%, 94.1%, and 85.3%, respectively.
  arXiv  Detail & Related papers  (2022-01-13T05:24:40Z)
• Learned super resolution ultrasound for improved breast lesion characterization [52.77024349608834]
  Super resolution ultrasound localization microscopy enables imaging of the microvasculature at the capillary level. In this work we use a deep neural network architecture that makes effective use of signal structure to address these challenges. By leveraging our trained network, the microvasculature structure is recovered in a short time, without prior PSF knowledge, and without requiring separability of the UCAs.
  arXiv  Detail & Related papers  (2021-07-12T09:04:20Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• DenseNet for Breast Tumor Classification in Mammographic Images [0.0]
  The aim of this study is to build a deep convolutional neural network method for automatic detection, segmentation, and classification of breast lesions in mammography images. Based on deep learning the Mask-CNN (RoIAlign) method was developed to features selection and extraction; and the classification was carried out by DenseNet architecture.
  arXiv  Detail & Related papers  (2021-01-24T03:30:59Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)
• Synthesizing lesions using contextual GANs improves breast cancer classification on mammograms [0.4297070083645048]
  We present a novel generative adversarial network (GAN) model for data augmentation that can realistically synthesize and remove lesions on mammograms. With self-attention and semi-supervised learning components, the U-net-based architecture can generate high resolution (256x256px) outputs.
  arXiv  Detail & Related papers  (2020-05-29T21:23:00Z)
• Co-Heterogeneous and Adaptive Segmentation from Multi-Source and Multi-Phase CT Imaging Data: A Study on Pathological Liver and Lesion Segmentation [48.504790189796836]
  We present a novel segmentation strategy, co-heterogenous and adaptive segmentation (CHASe) We propose a versatile framework that fuses appearance based semi-supervision, mask based adversarial domain adaptation, and pseudo-labeling. CHASe can further improve pathological liver mask Dice-Sorensen coefficients by ranges of $4.2% sim 9.4%$.
  arXiv  Detail & Related papers  (2020-05-27T06:58:39Z)
• Two-stage multi-scale breast mass segmentation for full mammogram analysis without user intervention [2.7490008316742096]
  Mammography is the primary imaging modality used for early detection and diagnosis of breast cancer. Among diverse types of breast abnormalities, masses are the most important clinical findings of breast carcinomas. We present a two-stage multi-scale pipeline that provides accurate mass contours from high-resolution full mammograms.
  arXiv  Detail & Related papers  (2020-02-27T13:16:22Z)
• An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization [45.00998416720726]
  We propose a framework to address the unique properties of medical images. This model first uses a low-capacity, yet memory-efficient, network on the whole image to identify the most informative regions. It then applies another higher-capacity network to collect details from chosen regions. Finally, it employs a fusion module that aggregates global and local information to make a final prediction.
  arXiv  Detail & Related papers  (2020-02-13T15:28:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.