Improving Representation Learning for Histopathologic Images with
Cluster Constraints
- URL: http://arxiv.org/abs/2310.12334v2
- Date: Tue, 14 Nov 2023 12:04:24 GMT
- Title: Improving Representation Learning for Histopathologic Images with
Cluster Constraints
- Authors: Weiyi Wu, Chongyang Gao, Joseph DiPalma, Soroush Vosoughi, Saeed
Hassanpour
- Abstract summary: Self-supervised learning (SSL) pretraining strategies are emerging as a viable alternative.
We introduce an SSL framework for transferable representation learning and semantically meaningful clustering.
Our approach outperforms common SSL methods in downstream classification and clustering tasks.
- Score: 31.426157660880673
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advances in whole-slide image (WSI) scanners and computational
capabilities have significantly propelled the application of artificial
intelligence in histopathology slide analysis. While these strides are
promising, current supervised learning approaches for WSI analysis come with
the challenge of exhaustively labeling high-resolution slides - a process that
is both labor-intensive and time-consuming. In contrast, self-supervised
learning (SSL) pretraining strategies are emerging as a viable alternative,
given that they don't rely on explicit data annotations. These SSL strategies
are quickly bridging the performance disparity with their supervised
counterparts. In this context, we introduce an SSL framework. This framework
aims for transferable representation learning and semantically meaningful
clustering by synergizing invariance loss and clustering loss in WSI analysis.
Notably, our approach outperforms common SSL methods in downstream
classification and clustering tasks, as evidenced by tests on the Camelyon16
and a pancreatic cancer dataset.
Related papers
- On the Discriminability of Self-Supervised Representation Learning [38.598160031349686]
Self-supervised learning (SSL) has recently achieved significant success in downstream visual tasks.
A notable gap still exists between SSL and supervised learning (SL), especially in complex downstream tasks.
arXiv Detail & Related papers (2024-07-18T14:18:03Z) - Reducing self-supervised learning complexity improves weakly-supervised
classification performance in computational pathology [0.0]
Self-supervised learning (SSL) methods allow for large-scale analyses on non-annotated data.
We investigated the complexity of SSL in relation to classification performance with the utilization of consumer-grade hardware.
Our experiments demonstrate that we can improve downstream classification performance whilst reducing SSL training duration by 90%.
arXiv Detail & Related papers (2024-03-07T14:56:06Z) - Making Self-supervised Learning Robust to Spurious Correlation via
Learning-speed Aware Sampling [26.444935219428036]
Self-supervised learning (SSL) has emerged as a powerful technique for learning rich representations from unlabeled data.
In real-world settings, spurious correlations between some attributes (e.g. race, gender and age) and labels for downstream tasks often exist.
We propose a learning-speed aware SSL (LA-SSL) approach, in which we sample each training data with a probability that is inversely related to its learning speed.
arXiv Detail & Related papers (2023-11-27T22:52:45Z) - Reverse Engineering Self-Supervised Learning [17.720366509919167]
Self-supervised learning (SSL) is a powerful tool in machine learning.
This paper presents an in-depth empirical analysis of SSL-trained representations.
arXiv Detail & Related papers (2023-05-24T23:15:28Z) - Spatiotemporal Self-supervised Learning for Point Clouds in the Wild [65.56679416475943]
We introduce an SSL strategy that leverages positive pairs in both the spatial and temporal domain.
We demonstrate the benefits of our approach via extensive experiments performed by self-supervised training on two large-scale LiDAR datasets.
arXiv Detail & Related papers (2023-03-28T18:06:22Z) - On Higher Adversarial Susceptibility of Contrastive Self-Supervised
Learning [104.00264962878956]
Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification.
It is still largely unknown if the nature of the representation induced by the two learning paradigms is similar.
We identify the uniform distribution of data representation over a unit hypersphere in the CSL representation space as the key contributor to this phenomenon.
We devise strategies that are simple, yet effective in improving model robustness with CSL training.
arXiv Detail & Related papers (2022-07-22T03:49:50Z) - Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of
Semi-Supervised Learning and Active Learning [60.26659373318915]
Active learning (AL) and semi-supervised learning (SSL) are two effective, but often isolated, means to alleviate the data-hungry problem.
We propose an innovative Inconsistency-based virtual aDvErial algorithm to further investigate SSL-AL's potential superiority.
Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithm.
arXiv Detail & Related papers (2022-06-07T13:28:43Z) - Using Representation Expressiveness and Learnability to Evaluate
Self-Supervised Learning Methods [61.49061000562676]
We introduce Cluster Learnability (CL) to assess learnability.
CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means.
We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
arXiv Detail & Related papers (2022-06-02T19:05:13Z) - On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks.
Despite these advances, the understanding of these methods is still relatively limited.
arXiv Detail & Related papers (2021-01-18T10:12:31Z) - Revisiting LSTM Networks for Semi-Supervised Text Classification via
Mixed Objective Function [106.69643619725652]
We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results.
We report state-of-the-art results for text classification task on several benchmark datasets.
arXiv Detail & Related papers (2020-09-08T21:55:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.