From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology

• URL: http://arxiv.org/abs/2204.05044v2
• Date: Tue, 9 May 2023 15:05:49 GMT
• Title: From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology
• Authors: Maximilian Springenberg, Annika Frommholz, Markus Wenzel, Eva Weicken, Jackie Ma, and Nils Strodthoff
• Abstract summary: We develop a new methodology to extensively evaluate a wide range of classification models. We thoroughly tested the models on five widely used histopathology datasets. We extend existing interpretability methods and systematically reveal insights of the models' classifications strategies.
• Score: 1.8947504307591034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While machine learning is currently transforming the field of histopathology, the domain lacks a comprehensive evaluation of state-of-the-art models based on essential but complementary quality requirements beyond a mere classification accuracy. In order to fill this gap, we developed a new methodology to extensively evaluate a wide range of classification models, including recent vision transformers, and convolutional neural networks such as: ConvNeXt, ResNet (BiT), Inception, ViT and Swin transformer, with and without supervised or self-supervised pretraining. We thoroughly tested the models on five widely used histopathology datasets containing whole slide images of breast, gastric, and colorectal cancer and developed a novel approach using an image-to-image translation model to assess the robustness of a cancer classification model against stain variations. Further, we extended existing interpretability methods to previously unstudied models and systematically reveal insights of the models' classifications strategies that can be transferred to future model architectures.

Related papers

• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Multi-Branch Generative Models for Multichannel Imaging with an Application to PET/CT Joint Reconstruction [42.95604565673447]
  This paper presents a proof-of-concept for learned synergistic reconstruction of medical images using multi-branch generative models. We demonstrate the efficacy of our approach on both Modified National Institute of Standards and Technology (MNIST) and positron emission tomography (PET)/computed tomography (CT) datasets. Despite challenges such as patch decomposition and model limitations, our results underscore the potential of generative models for enhancing medical imaging reconstruction.
  arXiv  Detail & Related papers  (2024-04-12T18:21:08Z)
• Prompt-Guided Adaptive Model Transformation for Whole Slide Image Classification [27.21493446754789]
  Multiple instance learning (MIL) has emerged as a popular method for classifying histopathology whole slide images (WSIs) We propose Prompt-guided Adaptive Model Transformation framework that seamlessly adapts pre-trained models to the specific characteristics of histopathology data. We rigorously evaluate our approach on two datasets, Camelyon16 and TCGA-NSCLC, showcasing substantial improvements across various MIL models.
  arXiv  Detail & Related papers  (2024-03-19T08:23:12Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• Domain-Specific Pre-training Improves Confidence in Whole Slide Image Classification [15.354256205808273]
  Whole Slide Images (WSIs) or histopathology images are used in digital pathology. WSIs pose great challenges to deep learning models for clinical diagnosis.
  arXiv  Detail & Related papers  (2023-02-20T08:42:06Z)
• Ultrasound Signal Processing: From Models to Deep Learning [64.56774869055826]
  Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Deep learning based methods, which are optimized in a data-driven fashion, have gained popularity. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning, as well as exploiting domain knowledge.
  arXiv  Detail & Related papers  (2022-04-09T13:04:36Z)
• Class-Aware Generative Adversarial Transformers for Medical Image Segmentation [39.14169989603906]
  We present CA-GANformer, a novel type of generative adversarial transformers, for medical image segmentation. First, we take advantage of the pyramid structure to construct multi-scale representations and handle multi-scale variations. We then design a novel class-aware transformer module to better learn the discriminative regions of objects with semantic structures.
  arXiv  Detail & Related papers  (2022-01-26T03:50:02Z)
• Evaluating Transformer based Semantic Segmentation Networks for Pathological Image Segmentation [2.7029872968576947]
  Histopathology has played an essential role in cancer diagnosis. Various CNN-based automated pathological image segmentation approaches have been developed in computer-assisted pathological image analysis. Transformer neural networks (Transformer) have shown the unique merit of capturing the global long distance dependencies across the entire image as a new deep learning paradigm.
  arXiv  Detail & Related papers  (2021-08-26T18:46:43Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [55.28436972267793]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.