Self-supervised dense representation learning for live-cell microscopy
with time arrow prediction
- URL: http://arxiv.org/abs/2305.05511v2
- Date: Wed, 26 Jul 2023 11:59:11 GMT
- Title: Self-supervised dense representation learning for live-cell microscopy
with time arrow prediction
- Authors: Benjamin Gallusser, Max Stieber, and Martin Weigert
- Abstract summary: We present a self-supervised method that learns dense image representations from raw, unlabeled live-cell microscopy videos.
We show that the resulting dense representations capture inherently time-asymmetric biological processes such as cell divisions on a pixel-level.
Our method outperforms supervised methods, particularly when only limited ground truth annotations are available.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: State-of-the-art object detection and segmentation methods for microscopy
images rely on supervised machine learning, which requires laborious manual
annotation of training data. Here we present a self-supervised method based on
time arrow prediction pre-training that learns dense image representations from
raw, unlabeled live-cell microscopy videos. Our method builds upon the task of
predicting the correct order of time-flipped image regions via a single-image
feature extractor followed by a time arrow prediction head that operates on the
fused features. We show that the resulting dense representations capture
inherently time-asymmetric biological processes such as cell divisions on a
pixel-level. We furthermore demonstrate the utility of these representations on
several live-cell microscopy datasets for detection and segmentation of
dividing cells, as well as for cell state classification. Our method
outperforms supervised methods, particularly when only limited ground truth
annotations are available as is commonly the case in practice. We provide code
at https://github.com/weigertlab/tarrow.
Related papers
- Unsupervised Learning of Object-Centric Embeddings for Cell Instance
Segmentation in Microscopy Images [3.039768384237206]
We introduce object-centric embeddings (OCEs)
OCEs embed image patches such that the offsets between patches cropped from the same object are preserved.
We show theoretically that OCEs can be learnt through a self-supervised task that predicts the spatial offset between image patches.
arXiv Detail & Related papers (2023-10-12T16:59:50Z) - MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments [72.6405488990753]
Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks.
We propose a single-stage and standalone method, MOCA, which unifies both desired properties.
We achieve new state-of-the-art results on low-shot settings and strong experimental results in various evaluation protocols.
arXiv Detail & Related papers (2023-07-18T15:46:20Z) - Seamless Iterative Semi-Supervised Correction of Imperfect Labels in
Microscopy Images [57.42492501915773]
In-vitro tests are an alternative to animal testing for the toxicity of medical devices.
Human fatigue plays a role in error making, making the use of deep learning appealing.
We propose Seamless Iterative Semi-Supervised correction of Imperfect labels (SISSI)
Our method successfully provides an adaptive early learning correction technique for object detection.
arXiv Detail & Related papers (2022-08-05T18:52:20Z) - Edge-Based Self-Supervision for Semi-Supervised Few-Shot Microscopy
Image Cell Segmentation [16.94384366469512]
We propose the prediction of edge-based maps for self-supervising the training of the unlabelled images.
In our experiments, we show that only a small number of annotated images, e.g. 10% of the original training set, is enough for our approach to reach similar performance as with the fully annotated databases on 1- to 10-shots.
arXiv Detail & Related papers (2022-08-03T14:35:00Z) - Learning multi-scale functional representations of proteins from
single-cell microscopy data [77.34726150561087]
We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information.
We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
arXiv Detail & Related papers (2022-05-24T00:00:07Z) - LEAD: Self-Supervised Landmark Estimation by Aligning Distributions of
Feature Similarity [49.84167231111667]
Existing works in self-supervised landmark detection are based on learning dense (pixel-level) feature representations from an image.
We introduce an approach to enhance the learning of dense equivariant representations in a self-supervised fashion.
We show that having such a prior in the feature extractor helps in landmark detection, even under drastically limited number of annotations.
arXiv Detail & Related papers (2022-04-06T17:48:18Z) - Cell segmentation from telecentric bright-field transmitted light
microscopic images using a Residual Attention U-Net: a case study on HeLa
line [0.0]
Living cell segmentation from bright-field light microscopic images is challenging due to the image complexity and temporal changes in the living cells.
Recently developed deep learning (DL)-based methods became popular in medical and microscopic image segmentation tasks due to their success and promising outcomes.
The main objective of this paper is to develop a deep learning, UNet-based method to segment the living cells of the HeLa line in bright-field transmitted light microscopy.
arXiv Detail & Related papers (2022-03-23T09:20:30Z) - Semi-supervised Cell Detection in Time-lapse Images Using Temporal
Consistency [10.20554144865699]
We propose a semi-supervised cell-detection method that effectively uses a time-lapse sequence with one labeled image and the other images unlabeled.
First, we train a cell-detection network with a one-labeled image and estimate the unlabeled images with the trained network.
We then select high-confidence positions from the estimations by tracking the detected cells from the labeled frame to those far from it.
arXiv Detail & Related papers (2021-07-19T06:40:47Z) - Instance Localization for Self-supervised Detection Pretraining [68.24102560821623]
We propose a new self-supervised pretext task, called instance localization.
We show that integration of bounding boxes into pretraining promotes better task alignment and architecture alignment for transfer learning.
Experimental results demonstrate that our approach yields state-of-the-art transfer learning results for object detection.
arXiv Detail & Related papers (2021-02-16T17:58:57Z) - Geography-Aware Self-Supervised Learning [79.4009241781968]
We show that due to their different characteristics, a non-trivial gap persists between contrastive and supervised learning on standard benchmarks.
We propose novel training methods that exploit the spatially aligned structure of remote sensing data.
Our experiments show that our proposed method closes the gap between contrastive and supervised learning on image classification, object detection and semantic segmentation for remote sensing.
arXiv Detail & Related papers (2020-11-19T17:29:13Z) - Cell Segmentation and Tracking using CNN-Based Distance Predictions and
a Graph-Based Matching Strategy [0.20999222360659608]
We present a method for the segmentation of touching cells in microscopy images.
By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process.
This representation is notably robust to annotation errors and shows promising results for the segmentation of microscopy images containing in the training data underrepresented or not included cell types.
arXiv Detail & Related papers (2020-04-03T11:55:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.