Feature Learning with Multi-Stage Vision Transformers on Inter-Modality HER2 Status Scoring and Tumor Classification on Whole Slides

• URL: http://arxiv.org/abs/2512.22335v1
• Date: Fri, 26 Dec 2025 17:45:09 GMT
• Title: Feature Learning with Multi-Stage Vision Transformers on Inter-Modality HER2 Status Scoring and Tumor Classification on Whole Slides
• Authors: Olaide N. Oyelade, Oliver Hoxey, Yulia Humrye,
• Abstract summary: We propose a single end-to-end pipeline using a system of vision transformers with HER2 status scoring on slide images.<n>A clinically inspired HER2 scoring mechanism is embedded in the pipeline and allows for automatic pixel-level annotation of 4-way HER2 scoring.<n>The proposed method accurately returns HER2-negative and HER2-positive.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The popular use of histopathology images, such as hematoxylin and eosin (H&E), has proven to be useful in detecting tumors. However, moving such cancer cases forward for treatment requires accurate on the amount of the human epidermal growth factor receptor 2 (HER2) protein expression. Predicting both the lower and higher levels of HER2 can be challenging. Moreover, jointly analyzing H&E and immunohistochemistry (IHC) stained images for HER2 scoring is difficult. Although several deep learning methods have been investigated to address the challenge of HER2 scoring, they suffer from providing a pixel-level localization of HER2 status. In this study, we propose a single end-to-end pipeline using a system of vision transformers with HER2 status scoring on whole slide images of WSIs. The method includes patch-wise processing of H&E WSIs for tumor localization. A novel mapping function is proposed to correspondingly identify correlated IHC WSIs regions with malignant regions on H&E. A clinically inspired HER2 scoring mechanism is embedded in the pipeline and allows for automatic pixel-level annotation of 4-way HER2 scoring (0, 1+, 2+, and 3+). Also, the proposed method accurately returns HER2-negative and HER2-positive. Privately curated datasets were collaboratively extracted from 13 different cases of WSIs of H&E and IHC. A thorough experiment is conducted on the proposed method. Results obtained showed a good classification accuracy during tumor localization. Also, a classification accuracy of 0.94 and a specificity of 0.933 were returned for the prediction of HER2 status, scoring in the 4-way methods. The applicability of the proposed pipeline was investigated using WSIs patches as comparable to human pathologists. Findings from the study showed the usability of jointly evaluated H&E and IHC images on end-to-end ViTs-based models for HER2 scoring

Related papers

• Transforming H&E images into IHC: A Variance-Penalized GAN for Precision Oncology [1.0972875392165036]
  This study proposes an advanced deep learning-based image translation framework to generate highfidelity IHC images from H&E-stained tissue samples.<n>By modifying the loss function of pyramid pix2pix, we mitigate mode collapse, a fundamental limitation in generative adversarial networks (GANs)<n>Our model particularly excels in translating HER2-positive (IHC 3+) images, which have remained challenging for existing methods due to their complex morphological variations.
  arXiv  Detail & Related papers  (2025-06-23T07:57:22Z)
• HER2 and FISH Status Prediction in Breast Biopsy H&E-Stained Images Using Deep Learning [3.1326332131745813]
  Hematoxylin and eosin (H&E) tumor stains are more widely available. accurately predicting HER2 status using H&E could reduce costs and expedite treatment selection. Deep Learning algorithms for H&E have shown effectiveness in predicting various cancer features and clinical outcomes.
  arXiv  Detail & Related papers  (2024-08-25T12:22:50Z)
• Interpretable HER2 scoring by evaluating clinical Guidelines through a weakly supervised, constrained Deep Learning Approach [0.0]
  This paper focuses on the interpretability of HER2 scoring by a pathologist. We propose a semi-automatic, two-stage deep learning approach that directly evaluates the clinical HER2 guidelines. We achieve a performance of 0.78 in F1-score on the test set while keeping our model interpretable for the pathologist.
  arXiv  Detail & Related papers  (2022-11-17T14:28:35Z)
• CoRe: An Automated Pipeline for The Prediction of Liver Resection Complexity from Preoperative CT Scans [53.561797148529664]
  Tumors located in critical positions are known to complexify liver resections. CoRe is an automated medical image processing pipeline for the prediction of postoperative LR complexity.
  arXiv  Detail & Related papers  (2022-10-15T15:29:24Z)
• Lung Cancer Lesion Detection in Histopathology Images Using Graph-Based Sparse PCA Network [93.22587316229954]
  We propose a graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E) We evaluate the performance of the proposed algorithm on H&E slides obtained from an SVM K-rasG12D lung cancer mouse model using precision/recall rates, F-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC)
  arXiv  Detail & Related papers  (2021-10-27T19:28:36Z)
• Automated Scoring System of HER2 in Pathological Images under the Microscope [3.689994643251581]
  We propose a HER2 automated scoring system that strictly follows the HER2 scoring guidelines. The proposed system will be embedded in our Thorough Eye platform for deployment in hospitals.
  arXiv  Detail & Related papers  (2021-10-21T01:38:35Z)
• Microscope Based HER2 Scoring System [37.747953235497256]
  We propose a real-time HER2 scoring system, which follows the HER2 scoring guidelines to complete diagnosis. Unlike the previous scoring systems based on whole-slide imaging, our HER2 scoring system is integrated into an augmented reality microscope.
  arXiv  Detail & Related papers  (2020-09-15T01:44:39Z)
• Multiple Sclerosis Lesion Activity Segmentation with Attention-Guided Two-Path CNNs [49.32653090178743]
  convolutional neural networks (CNNs) are studied for lesion activity segmentation from two time points. CNNs are designed and evaluated that combine the information from two points in different ways. It is demonstrated that deep learning-based methods outperform classic approaches.
  arXiv  Detail & Related papers  (2020-08-05T08:49:20Z)
• Segmentation of the Myocardium on Late-Gadolinium Enhanced MRI based on 2.5 D Residual Squeeze and Excitation Deep Learning Model [55.09533240649176]
  The aim of this work is to develop an accurate automatic segmentation method based on deep learning models for the myocardial borders on LGE-MRI. A total number of 320 exams (with a mean number of 6 slices per exam) were used for training and 28 exams used for testing. The performance analysis of the proposed ensemble model in the basal and middle slices was similar as compared to intra-observer study and slightly lower at apical slices.
  arXiv  Detail & Related papers  (2020-05-27T20:44:38Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.