PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and
IHC Stains
- URL: http://arxiv.org/abs/2312.09894v1
- Date: Fri, 15 Dec 2023 15:45:52 GMT
- Title: PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and
IHC Stains
- Authors: Shengyi Hua, Fang Yan, Tianle Shen, Xiaofan Zhang
- Abstract summary: We present PathoDuet, a series of pretrained models on histopathology images, and a new self-supervised learning framework in histopathology.
The framework is featured by a newly-introduced pretext token and later task raisers to explicitly utilize certain relations between images.
Based on this, two pretext tasks, cross-scale positioning and cross-stain transferring, are designed to pretrain the model on Hematoxylin and Eosin images.
- Score: 2.77305170426095
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large amounts of digitized histopathological data display a promising future
for developing pathological foundation models via self-supervised learning
methods. Foundation models pretrained with these methods serve as a good basis
for downstream tasks. However, the gap between natural and histopathological
images hinders the direct application of existing methods. In this work, we
present PathoDuet, a series of pretrained models on histopathological images,
and a new self-supervised learning framework in histopathology. The framework
is featured by a newly-introduced pretext token and later task raisers to
explicitly utilize certain relations between images, like multiple
magnifications and multiple stains. Based on this, two pretext tasks,
cross-scale positioning and cross-stain transferring, are designed to pretrain
the model on Hematoxylin and Eosin (H\&E) images and transfer the model to
immunohistochemistry (IHC) images, respectively. To validate the efficacy of
our models, we evaluate the performance over a wide variety of downstream
tasks, including patch-level colorectal cancer subtyping and whole slide image
(WSI)-level classification in H\&E field, together with expression level
prediction of IHC marker and tumor identification in IHC field. The
experimental results show the superiority of our models over most tasks and the
efficacy of proposed pretext tasks. The codes and models are available at
https://github.com/openmedlab/PathoDuet.
Related papers
- MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep
Models for X-ray Images of Multiple Body Parts [63.30352394004674]
Multi-task Self-super-vised Continual Learning (MUSCLE) is a novel self-supervised pre-training pipeline for medical imaging tasks.
MUSCLE aggregates X-rays collected from multiple body parts for representation learning, and adopts a well-designed continual learning procedure.
We evaluate MUSCLE using 9 real-world X-ray datasets with various tasks, including pneumonia classification, skeletal abnormality classification, lung segmentation, and tuberculosis (TB) detection.
arXiv Detail & Related papers (2023-10-03T12:19:19Z) - Nucleus-aware Self-supervised Pretraining Using Unpaired Image-to-image
Translation for Histopathology Images [3.8391355786589805]
We propose a novel nucleus-aware self-supervised pretraining framework for histopathology images.
The framework aims to capture the nuclear morphology and distribution information through unpaired image-to-image translation.
The experiments on 7 datasets show that the proposed pretraining method outperforms supervised ones on Kather classification, multiple instance learning, and 5 dense-prediction tasks.
arXiv Detail & Related papers (2023-09-14T02:31:18Z) - Towards a Visual-Language Foundation Model for Computational Pathology [5.72536252929528]
We introduce CONtrastive learning from Captions for Histopathology (CONCH)
CONCH is a visual-language foundation model developed using diverse sources of histopathology images, biomedical text, and task-agnostic pretraining.
It is evaluated on a suite of 13 diverse benchmarks, achieving state-of-the-art performance on histology image classification, segmentation, captioning, text-to-image and image-to-text retrieval.
arXiv Detail & Related papers (2023-07-24T16:13:43Z) - The Whole Pathological Slide Classification via Weakly Supervised
Learning [7.313528558452559]
We introduce two pathological priors: nuclear disease of cells and spatial correlation of pathological tiles.
We propose a data augmentation method that utilizes stain separation during extractor training.
We then describe the spatial relationships between the tiles using an adjacency matrix.
By integrating these two views, we designed a multi-instance framework for analyzing H&E-stained tissue images.
arXiv Detail & Related papers (2023-07-12T16:14:23Z) - MedFMC: A Real-world Dataset and Benchmark For Foundation Model
Adaptation in Medical Image Classification [41.16626194300303]
Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications.
Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples.
Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks.
arXiv Detail & Related papers (2023-06-16T01:46:07Z) - Domain-Specific Pre-training Improves Confidence in Whole Slide Image
Classification [15.354256205808273]
Whole Slide Images (WSIs) or histopathology images are used in digital pathology.
WSIs pose great challenges to deep learning models for clinical diagnosis.
arXiv Detail & Related papers (2023-02-20T08:42:06Z) - Learning Discriminative Representation via Metric Learning for
Imbalanced Medical Image Classification [52.94051907952536]
We propose embedding metric learning into the first stage of the two-stage framework specially to help the feature extractor learn to extract more discriminative feature representations.
Experiments mainly on three medical image datasets show that the proposed approach consistently outperforms existing onestage and two-stage approaches.
arXiv Detail & Related papers (2022-07-14T14:57:01Z) - A multi-stage machine learning model on diagnosis of esophageal
manometry [50.591267188664666]
The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage.
This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
arXiv Detail & Related papers (2021-06-25T20:09:23Z) - Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for
Thoracic Disease Identification [83.6017225363714]
deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance.
For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming.
In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
arXiv Detail & Related papers (2021-02-26T02:29:30Z) - Deep Low-Shot Learning for Biological Image Classification and
Visualization from Limited Training Samples [52.549928980694695]
In situ hybridization (ISH) gene expression pattern images from the same developmental stage are compared.
labeling training data with precise stages is very time-consuming even for biologists.
We propose a deep two-step low-shot learning framework to accurately classify ISH images using limited training images.
arXiv Detail & Related papers (2020-10-20T06:06:06Z) - Retinopathy of Prematurity Stage Diagnosis Using Object Segmentation and
Convolutional Neural Networks [68.96150598294072]
Retinopathy of Prematurity (ROP) is an eye disorder primarily affecting premature infants with lower weights.
It causes proliferation of vessels in the retina and could result in vision loss and, eventually, retinal detachment, leading to blindness.
In recent years, there has been a significant effort to automate the diagnosis using deep learning.
This paper builds upon the success of previous models and develops a novel architecture, which combines object segmentation and convolutional neural networks (CNN)
Our proposed system first trains an object segmentation model to identify the demarcation line at a pixel level and adds the resulting mask as an additional "color" channel in
arXiv Detail & Related papers (2020-04-03T14:07:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.