Unsupervised Fish Trajectory Tracking and Segmentation

• URL: http://arxiv.org/abs/2208.10662v1
• Date: Tue, 23 Aug 2022 01:01:27 GMT
• Title: Unsupervised Fish Trajectory Tracking and Segmentation
• Authors: Alzayat Saleh, Marcus Sheaves, Dean Jerry, Mostafa Rahimi Azghadi
• Abstract summary: We propose a three-stage framework for robust fish tracking and segmentation. The first stage is an optical flow model, which generates the pseudo labels using spatial and temporal consistency between frames. In the second stage, a self-supervised model refines the pseudo-labels incrementally. In the third stage, the refined labels are used to train a segmentation network.
• Score: 2.1028463367241033
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: DNN for fish tracking and segmentation based on high-quality labels is expensive. Alternative unsupervised approaches rely on spatial and temporal variations that naturally occur in video data to generate noisy pseudo-ground-truth labels. These pseudo-labels are used to train a multi-task deep neural network. In this paper, we propose a three-stage framework for robust fish tracking and segmentation, where the first stage is an optical flow model, which generates the pseudo labels using spatial and temporal consistency between frames. In the second stage, a self-supervised model refines the pseudo-labels incrementally. In the third stage, the refined labels are used to train a segmentation network. No human annotations are used during the training or inference. Extensive experiments are performed to validate our method on three public underwater video datasets and to demonstrate that it is highly effective for video annotation and segmentation. We also evaluate the robustness of our framework to different imaging conditions and discuss the limitations of our current implementation.

Related papers

• TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detection [59.498894868956306]
  Pseudo-labeling approaches to semi-supervised learning adopt a teacher-student framework. We leverage pre-trained motion-forecasting models to generate object trajectories on pseudo-labeled data. Our approach improves pseudo-label quality in two distinct manners.
  arXiv  Detail & Related papers  (2024-09-17T05:35:00Z)
• Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment [62.73503467108322]
  This topic is widely studied in 3D point cloud segmentation due to the difficulty of annotating point clouds densely. Until recently, pseudo-labels have been widely employed to facilitate training with limited ground-truth labels. Existing pseudo-labeling approaches could suffer heavily from the noises and variations in unlabelled data. We propose a novel learning strategy to regularize the pseudo-labels generated for training, thus effectively narrowing the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2024-08-29T13:31:15Z)
• Label-Efficient 3D Brain Segmentation via Complementary 2D Diffusion Models with Orthogonal Views [10.944692719150071]
  We propose a novel 3D brain segmentation approach using complementary 2D diffusion models. Our goal is to achieve reliable segmentation quality without requiring complete labels for each individual subject.
  arXiv  Detail & Related papers  (2024-07-17T06:14:53Z)
• WildScenes: A Benchmark for 2D and 3D Semantic Segmentation in Large-scale Natural Environments [33.25040383298019]
  $WildScenes$ is a bi-modal benchmark dataset consisting of high-resolution 2D images and dense 3D LiDAR point clouds. The data is trajectory-centric with accurate localization and globally aligned point clouds. Our 3D semantic labels are obtained via an efficient, automated process that transfers the human-annotated 2D labels from multiple views into 3D point cloud sequences.
  arXiv  Detail & Related papers  (2023-12-23T22:27:40Z)
• Beyond the Label Itself: Latent Labels Enhance Semi-supervised Point Cloud Panoptic Segmentation [46.01433705072047]
  We find two types of latent labels behind the displayed label embedded in LiDAR and image data. We propose a novel augmentation, Cylinder-Mix, which is able to augment more yet reliable samples for training. We also propose the Instance Position-scale Learning (IPSL) Module to learn and fuse the information of instance position and scale.
  arXiv  Detail & Related papers  (2023-12-13T15:56:24Z)
• Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior [100.98123802027847]
  We investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. We generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation. Our method achieves new state-of-the-art performance in self-supervised scene flow learning.
  arXiv  Detail & Related papers  (2023-10-17T14:06:55Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning [59.64695628433855]
  We propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images. Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels. Our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.
  arXiv  Detail & Related papers  (2022-09-16T07:59:04Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.