ASIST: Annotation-free synthetic instance segmentation and tracking for microscope video analysis

• URL: http://arxiv.org/abs/2011.01009v1
• Date: Mon, 2 Nov 2020 14:39:26 GMT
• Title: ASIST: Annotation-free synthetic instance segmentation and tracking for microscope video analysis
• Authors: Quan Liu, Isabella M. Gaeta, Mengyang Zhao, Ruining Deng, Aadarsh Jha, Bryan A. Millis, Anita Mahadevan-Jansen, Matthew J. Tyska, Yuankai Huo
• Abstract summary: We propose a novel annotation-free synthetic instance segmentation and tracking (ASIST) algorithm for analyzing microscope videos of sub-cellular microvilli. From the experimental results, the proposed annotation-free method achieved superior performance compared with supervised learning.
• Score: 8.212196747588361
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance object segmentation and tracking provide comprehensive quantification of objects across microscope videos. The recent single-stage pixel-embedding based deep learning approach has shown its superior performance compared with "segment-then-associate" two-stage solutions. However, one major limitation of applying a supervised pixel-embedding based method to microscope videos is the resource-intensive manual labeling, which involves tracing hundreds of overlapped objects with their temporal associations across video frames. Inspired by the recent generative adversarial network (GAN) based annotation-free image segmentation, we propose a novel annotation-free synthetic instance segmentation and tracking (ASIST) algorithm for analyzing microscope videos of sub-cellular microvilli. The contributions of this paper are three-fold: (1) proposing a new annotation-free video analysis paradigm is proposed. (2) aggregating the embedding based instance segmentation and tracking with annotation-free synthetic learning as a holistic framework; and (3) to the best of our knowledge, this is first study to investigate microvilli instance segmentation and tracking using embedding based deep learning. From the experimental results, the proposed annotation-free method achieved superior performance compared with supervised learning.

Related papers

• What is Point Supervision Worth in Video Instance Segmentation? [119.71921319637748]
  Video instance segmentation (VIS) is a challenging vision task that aims to detect, segment, and track objects in videos. We reduce the human annotations to only one point for each object in a video frame during training, and obtain high-quality mask predictions close to fully supervised models. Comprehensive experiments on three VIS benchmarks demonstrate competitive performance of the proposed framework, nearly matching fully supervised methods.
  arXiv  Detail & Related papers  (2024-04-01T17:38:25Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Exemplar Learning for Medical Image Segmentation [38.61378161105941]
  We propose an Exemplar Learning-based Synthesis Net (ELSNet) framework for medical image segmentation. ELSNet introduces two new modules for image segmentation: an exemplar-guided synthesis module and a pixel-prototype based contrastive embedding module. We conduct experiments on several organ segmentation datasets and present an in-depth analysis.
  arXiv  Detail & Related papers  (2022-04-03T00:10:06Z)
• Joint Inductive and Transductive Learning for Video Object Segmentation [107.32760625159301]
  Semi-supervised object segmentation is a task of segmenting the target object in a video sequence given only a mask in the first frame. Most previous best-performing methods adopt matching-based transductive reasoning or online inductive learning. We propose to integrate transductive and inductive learning into a unified framework to exploit complement between them for accurate and robust video object segmentation.
  arXiv  Detail & Related papers  (2021-08-08T16:25:48Z)
• Semi-TCL: Semi-Supervised Track Contrastive Representation Learning [40.31083437957288]
  We design a new instance-to-track matching objective to learn appearance embedding. It compares a candidate detection to the embedding of the tracks persisted in the tracker. We implement this learning objective in a unified form following the spirit of constrastive loss.
  arXiv  Detail & Related papers  (2021-07-06T05:23:30Z)
• Learning to Track Instances without Video Annotations [85.9865889886669]
  We introduce a novel semi-supervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences. We show that even when only trained with images, the learned feature representation is robust to instance appearance variations. In addition, we integrate this module into single-stage instance segmentation and pose estimation frameworks.
  arXiv  Detail & Related papers  (2021-04-01T06:47:41Z)
• Towards Annotation-free Instance Segmentation and Tracking with Adversarial Simulations [5.434831972326107]
  In computer vision, annotated training data with consistent segmentation and tracking is resource intensive. adversarial simulations have provided successful solutions in computer vision to train real-world self-driving systems. This paper proposes an annotation-free synthetic instance segmentation and tracking (ASIST) method with adversarial simulation and single-stage pixel-embedding based learning.
  arXiv  Detail & Related papers  (2021-01-03T07:04:13Z)
• Unsupervised Learning of Video Representations via Dense Trajectory Clustering [86.45054867170795]
  This paper addresses the task of unsupervised learning of representations for action recognition in videos. We first propose to adapt two top performing objectives in this class - instance recognition and local aggregation. We observe promising performance, but qualitative analysis shows that the learned representations fail to capture motion patterns.
  arXiv  Detail & Related papers  (2020-06-28T22:23:03Z)
• Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos [23.153335327822685]
  We learn a motion-centric representation of surgical video demonstrations by grouping them into action segments/sub-goals/options. We present Motion2Vec, an algorithm that learns a deep embedding feature space from video observations. We demonstrate the use of this representation to imitate surgical suturing motions from publicly available videos of the JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-05-31T15:46:01Z)
• Motion-supervised Co-Part Segmentation [88.40393225577088]
  We propose a self-supervised deep learning method for co-part segmentation. Our approach develops the idea that motion information inferred from videos can be leveraged to discover meaningful object parts.
  arXiv  Detail & Related papers  (2020-04-07T09:56:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.