HyFormer-Net: A Synergistic CNN-Transformer with Interpretable Multi-Scale Fusion for Breast Lesion Segmentation and Classification in Ultrasound Images

• URL: http://arxiv.org/abs/2511.01013v1
• Date: Sun, 02 Nov 2025 17:06:12 GMT
• Title: HyFormer-Net: A Synergistic CNN-Transformer with Interpretable Multi-Scale Fusion for Breast Lesion Segmentation and Classification in Ultrasound Images
• Abstract summary: HyFormer-Net is a hybrid CNN-Transformer for simultaneous segmentation and classification with intrinsic interpretability.<n>On the BUSI dataset, HyFormer-Net achieves Dice Score 0.761 +/- 0.072 and accuracy 93.2%, outperforming U-Net, Attention U-Net, and TransUNet.<n>We conduct the first cross-dataset generalization study for hybrid CNN-Transformers in breast ultrasound.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: B-mode ultrasound for breast cancer diagnosis faces challenges: speckle, operator dependency, and indistinct boundaries. Existing deep learning suffers from single-task learning, architectural constraints (CNNs lack global context, Transformers local features), and black-box decision-making. These gaps hinder clinical adoption. We propose HyFormer-Net, a hybrid CNN-Transformer for simultaneous segmentation and classification with intrinsic interpretability. Its dual-branch encoder integrates EfficientNet-B3 and Swin Transformer via multi-scale hierarchical fusion blocks. An attention-gated decoder provides precision and explainability. We introduce dual-pipeline interpretability: (1) intrinsic attention validation with quantitative IoU verification (mean: 0.86), and (2) Grad-CAM for classification reasoning. On the BUSI dataset, HyFormer-Net achieves Dice Score 0.761 +/- 0.072 and accuracy 93.2%, outperforming U-Net, Attention U-Net, and TransUNet. Malignant Recall of 92.1 +/- 2.2% ensures minimal false negatives. Ensemble modeling yields exceptional Dice 90.2%, accuracy 99.5%, and perfect 100% Malignant Recall, eliminating false negatives. Ablation studies confirm multi-scale fusion contributes +16.8% Dice and attention gates add +5.9%. Crucially, we conduct the first cross-dataset generalization study for hybrid CNN-Transformers in breast ultrasound. Zero-shot transfer fails (Dice: 0.058), confirming domain shift. However, progressive fine-tuning with only 10% target-domain data (68 images) recovers 92.5% performance. With 50% data, our model achieves 77.3% Dice, exceeding source-domain performance (76.1%) and demonstrating true generalization.

Related papers

• Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models [0.8590539335179757]
  This research paper proposes utilizing unsupervised Denoising Diffusion Probabilistic Models (DDPMs) to extract anatomical features.<n>We train a DDPM on 21 unlabeled abdominal CT scans to learn structural representations, transferring the encoder weights to a downstream segmentation task.<n>Using unlabeled images for diffusion-based pretraining successfully embeds robust anatomical features prior to human supervision.
  arXiv  Detail & Related papers  (2026-08-26T12:10:56Z)
• Post-Operative Glioma Segmentation via Loss Stabilization, Normalization and Subspace Attention [46.03321798937855]
  We show that the standard Generalized Dice Loss (GDL) is unstable under domain shift.<n>We propose a Subspace-Aware Class Attention (SACA) module that re-calibrates the bottleneck features and raises Enhancing Tumor (ET) sensitivity by 8%.
  arXiv  Detail & Related papers  (2026-07-23T09:59:30Z)
• Cross-Stage Attention Multi-Expert Network for Radiologist-Inspired Breast Ultrasound Diagnosis [6.727825549527392]
  This paper proposes the Cross-Stage Attention Mixture-of-Experts Network (CSA-MoE-Net) to improve feature representation and classification accuracy.<n>It adopts a Cross-Stage Attention-enhanced ResNet-18 as the backbone, in which the Cross-Stage Attention module adaptively recalibrates multi-level features.<n> Experiments on a balanced dataset of 2,129 breast ultrasound images show that, averaged over 20 independent runs, the model achieves an accuracy of 96.33%, precision of 94.09%, recall of 98.53%, F1-score of 96.25%, and
  arXiv  Detail & Related papers  (2026-05-25T07:20:13Z)
• HADS-Net:A Hybrid Attention-Augmented Dual-Stream Network with Physics-Informed Augmentation for Breast Ultrasound Image Classification [0.04999814847776097]
  HADS-Net is a Hybrid Attention-Augmented Dual-Stream Network.<n>It exploits global texture and local boundary cues through two parallel pathways.<n>It achieves 96.58% accuracy, macro ROC-AUC of 0.9978, macro F1 of 0.9654, and per-class F1-scores of 0.970, 0.951, and 0.976 for benign, malignant, and normal.
  arXiv  Detail & Related papers  (2026-05-19T22:16:24Z)
• Performance of a Deep Learning-Based Segmentation Model for Pancreatic Tumors on Public Endoscopic Ultrasound Datasets [2.925528117330222]
  Pancreatic cancer is one of the most aggressive cancers, with poor survival rates.<n>This study evaluates a Vision Transformer-based deep learning segmentation model for pancreatic tumors.
  arXiv  Detail & Related papers  (2026-01-09T16:48:50Z)
• Validating Vision Transformers for Otoscopy: Performance and Data-Leakage Effects [42.465094107111646]
  This study evaluates the efficacy of vision transformer models, specifically Swin transformers, in enhancing the diagnostic accuracy of ear diseases.<n>The research utilised a real-world dataset from the Department of Otolaryngology at the Clinical Hospital of the Universidad de Chile.
  arXiv  Detail & Related papers  (2025-11-06T23:20:37Z)
• MSRANetV2: An Explainable Deep Learning Architecture for Multi-class Classification of Colorectal Histopathological Images [3.4859776888706233]
  Colorectal cancer (CRC) is a leading worldwide cause of cancer-related mortality.<n>Deep learning algorithms have become a powerful approach in enhancing diagnostic precision and efficiency.<n>We propose a convolutional neural network architecture named MSRANetV2, specially optimized for the classification of colorectal tissue images.
  arXiv  Detail & Related papers  (2025-10-28T07:22:34Z)
• Uncertainty-aware Cross-training for Semi-supervised Medical Image Segmentation [45.96892342675963]
  We propose an Uncertainty-aware Cross-training framework for semi-supervised medical image (UC-Seg)<n>Our method achieves superior segmentation accuracy and generalization performance compared to other state-of-the-art semi-supervised methods.
  arXiv  Detail & Related papers  (2025-08-12T15:28:10Z)
• A Semantic Segmentation Algorithm for Pleural Effusion Based on DBIF-AUNet [22.657295396752023]
  Pleural effusion semantic segmentation can significantly enhance the accuracy and timeliness of clinical diagnosis and treatment.<n>Existing methods often struggle with diverse image variations and complex edges.<n>We propose the Dual-Branch Interactive Fusion Attention model (DBIF-AUNet) to address these challenges.
  arXiv  Detail & Related papers  (2025-08-08T10:14:51Z)
• FUTransUNet-GradCAM: A Hybrid Transformer-U-Net with Self-Attention and Explainable Visualizations for Foot Ulcer Segmentation [0.0]
  Automated segmentation of diabetic foot ulcers (DFUs) plays a critical role in clinical diagnosis, therapeutic planning, and longitudinal wound monitoring.<n>Traditional convolutional neural networks (CNNs) provide strong localization capabilities but struggle to model long-range spatial dependencies.<n>We propose FUTransUNet, a hybrid architecture that integrates the global attention mechanism of Vision Transformers (ViTs) into the U-Net framework.
  arXiv  Detail & Related papers  (2025-08-04T11:05:14Z)
• Advanced U-Net Architectures with CNN Backbones for Automated Lung Cancer Detection and Segmentation in Chest CT Images [0.0]
  This study investigates the effectiveness of U-Net architectures integrated with various convolutional neural network (CNN) backbones for automated lung cancer detection and segmentation in chest CT images.<n>U-Net models were developed with three CNN backbones: ResNet50, VGG16, and Xception, to segment lung regions.<n>For classification, the CNN model using U-Net with Xception achieved 99.1 percent accuracy, 99.74 percent recall, and 99.42 percent F1-score.
  arXiv  Detail & Related papers  (2025-07-14T04:08:33Z)
• Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network [63.845552349914186]
  Capturing global contextual information plays a critical role in breast ultrasound (BUS) image classification. Vision Transformers have an improved capability of capturing global contextual information but may distort the local image patterns due to the tokenization operations. In this study, we proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN, designed to perform BUS tumor classification and segmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:19:32Z)
• Scaling Up 3D Kernels with Bayesian Frequency Re-parameterization for Medical Image Segmentation [25.62587471067468]
  RepUX-Net is a pure CNN architecture with a simple large kernel block design. Inspired by the spatial frequency in the human visual system, we extend to vary the kernel convergence into element-wise setting.
  arXiv  Detail & Related papers  (2023-03-10T08:38:34Z)
• Hybrid Window Attention Based Transformer Architecture for Brain Tumor Segmentation [28.650980942429726]
  We propose a volumetric vision transformer that follows two windowing strategies in attention for extracting fine features. We trained and evaluated network architecture on the FeTS Challenge 2022 dataset. Our performance on the online validation dataset is as follows: Dice Similarity Score of 81.71%, 91.38% and 85.40%.
  arXiv  Detail & Related papers  (2022-09-16T03:55:48Z)
• Automated Pulmonary Embolism Detection from CTPA Images Using an End-to-End Convolutional Neural Network [31.58557856188164]
  This study presents an end-to-end trainable convolutional neural network (CNN) for detecting pulmonary embolisms (PEs) Our system achieves a sensitivity of 63.2%, 78.9% and 86.8% at 2 false positives per volume at 0mm, 2mm and 5mm localization error.
  arXiv  Detail & Related papers  (2021-11-10T03:01:55Z)
• Vision Transformers for femur fracture classification [59.99241204074268]
  The Vision Transformer (ViT) was able to correctly predict 83% of the test images. Good results were obtained in sub-fractures with the largest and richest dataset ever.
  arXiv  Detail & Related papers  (2021-08-07T10:12:42Z)
• D2A U-Net: Automatic Segmentation of COVID-19 Lesions from CT Slices with Dilated Convolution and Dual Attention Mechanism [9.84838467721235]
  We propose a dilated dual attention U-Net (D2A U-Net) for COVID-19 lesion segmentation in CT slices based on dilated convolution and a novel dual attention mechanism. Our experiment results have shown that by introducing dilated convolution and dual attention mechanism, the number of false positives is significantly reduced.
  arXiv  Detail & Related papers  (2021-02-10T01:21:59Z)
• An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation [53.425900196763756]
  We propose a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen.
  arXiv  Detail & Related papers  (2020-12-06T18:55:07Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.