Self-supervised dense representation learning for live-cell microscopy with time arrow prediction

• URL: http://arxiv.org/abs/2305.05511v2
• Date: Wed, 26 Jul 2023 11:59:11 GMT
• Title: Self-supervised dense representation learning for live-cell microscopy with time arrow prediction
• Authors: Benjamin Gallusser, Max Stieber, and Martin Weigert
• Abstract summary: We present a self-supervised method that learns dense image representations from raw, unlabeled live-cell microscopy videos. We show that the resulting dense representations capture inherently time-asymmetric biological processes such as cell divisions on a pixel-level. Our method outperforms supervised methods, particularly when only limited ground truth annotations are available.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art object detection and segmentation methods for microscopy images rely on supervised machine learning, which requires laborious manual annotation of training data. Here we present a self-supervised method based on time arrow prediction pre-training that learns dense image representations from raw, unlabeled live-cell microscopy videos. Our method builds upon the task of predicting the correct order of time-flipped image regions via a single-image feature extractor followed by a time arrow prediction head that operates on the fused features. We show that the resulting dense representations capture inherently time-asymmetric biological processes such as cell divisions on a pixel-level. We furthermore demonstrate the utility of these representations on several live-cell microscopy datasets for detection and segmentation of dividing cells, as well as for cell state classification. Our method outperforms supervised methods, particularly when only limited ground truth annotations are available as is commonly the case in practice. We provide code at https://github.com/weigertlab/tarrow.

Related papers

• Self-supervised Representation Learning for Cell Event Recognition through Time Arrow Prediction [23.611375087515963]
  Deep-learning or segmentation tracking methods rely on large amount of high quality annotations to work effectively. In this work, we explore an alternative solution: using feature annotations from self-supervised representation learning (SSRL) for the downstream task of cell event recognition. Our analysis also provides insight into applications of the SSRL using TAP in live-cell microscopy.
  arXiv  Detail & Related papers  (2024-11-06T13:54:26Z)
• DiffKillR: Killing and Recreating Diffeomorphisms for Cell Annotation in Dense Microscopy Images [105.46086313858062]
  We introduce DiffKillR, a novel framework that reframes cell annotation as the combination of archetype matching and image registration tasks. We will discuss the theoretical properties of DiffKillR and validate it on three microscopy tasks, demonstrating its advantages over existing supervised, semi-supervised, and unsupervised methods.
  arXiv  Detail & Related papers  (2024-10-04T00:38:29Z)
• Unsupervised Learning of Object-Centric Embeddings for Cell Instance Segmentation in Microscopy Images [3.039768384237206]
  We introduce object-centric embeddings (OCEs) OCEs embed image patches such that the offsets between patches cropped from the same object are preserved. We show theoretically that OCEs can be learnt through a self-supervised task that predicts the spatial offset between image patches.
  arXiv  Detail & Related papers  (2023-10-12T16:59:50Z)
• MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments [72.6405488990753]
  Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks. We propose a single-stage and standalone method, MOCA, which unifies both desired properties. We achieve new state-of-the-art results on low-shot settings and strong experimental results in various evaluation protocols.
  arXiv  Detail & Related papers  (2023-07-18T15:46:20Z)
• Seamless Iterative Semi-Supervised Correction of Imperfect Labels in Microscopy Images [57.42492501915773]
  In-vitro tests are an alternative to animal testing for the toxicity of medical devices. Human fatigue plays a role in error making, making the use of deep learning appealing. We propose Seamless Iterative Semi-Supervised correction of Imperfect labels (SISSI) Our method successfully provides an adaptive early learning correction technique for object detection.
  arXiv  Detail & Related papers  (2022-08-05T18:52:20Z)
• Edge-Based Self-Supervision for Semi-Supervised Few-Shot Microscopy Image Cell Segmentation [16.94384366469512]
  We propose the prediction of edge-based maps for self-supervising the training of the unlabelled images. In our experiments, we show that only a small number of annotated images, e.g. 10% of the original training set, is enough for our approach to reach similar performance as with the fully annotated databases on 1- to 10-shots.
  arXiv  Detail & Related papers  (2022-08-03T14:35:00Z)
• LEAD: Self-Supervised Landmark Estimation by Aligning Distributions of Feature Similarity [49.84167231111667]
  Existing works in self-supervised landmark detection are based on learning dense (pixel-level) feature representations from an image. We introduce an approach to enhance the learning of dense equivariant representations in a self-supervised fashion. We show that having such a prior in the feature extractor helps in landmark detection, even under drastically limited number of annotations.
  arXiv  Detail & Related papers  (2022-04-06T17:48:18Z)
• Cell segmentation from telecentric bright-field transmitted light microscopic images using a Residual Attention U-Net: a case study on HeLa line [0.0]
  Living cell segmentation from bright-field light microscopic images is challenging due to the image complexity and temporal changes in the living cells. Recently developed deep learning (DL)-based methods became popular in medical and microscopic image segmentation tasks due to their success and promising outcomes. The main objective of this paper is to develop a deep learning, UNet-based method to segment the living cells of the HeLa line in bright-field transmitted light microscopy.
  arXiv  Detail & Related papers  (2022-03-23T09:20:30Z)
• Semi-supervised Cell Detection in Time-lapse Images Using Temporal Consistency [10.20554144865699]
  We propose a semi-supervised cell-detection method that effectively uses a time-lapse sequence with one labeled image and the other images unlabeled. First, we train a cell-detection network with a one-labeled image and estimate the unlabeled images with the trained network. We then select high-confidence positions from the estimations by tracking the detected cells from the labeled frame to those far from it.
  arXiv  Detail & Related papers  (2021-07-19T06:40:47Z)
• Instance Localization for Self-supervised Detection Pretraining [68.24102560821623]
  We propose a new self-supervised pretext task, called instance localization. We show that integration of bounding boxes into pretraining promotes better task alignment and architecture alignment for transfer learning. Experimental results demonstrate that our approach yields state-of-the-art transfer learning results for object detection.
  arXiv  Detail & Related papers  (2021-02-16T17:58:57Z)
• Geography-Aware Self-Supervised Learning [79.4009241781968]
  We show that due to their different characteristics, a non-trivial gap persists between contrastive and supervised learning on standard benchmarks. We propose novel training methods that exploit the spatially aligned structure of remote sensing data. Our experiments show that our proposed method closes the gap between contrastive and supervised learning on image classification, object detection and semantic segmentation for remote sensing.
  arXiv  Detail & Related papers  (2020-11-19T17:29:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.