Unsupervised Fish Trajectory Tracking and Segmentation
- URL: http://arxiv.org/abs/2208.10662v1
- Date: Tue, 23 Aug 2022 01:01:27 GMT
- Title: Unsupervised Fish Trajectory Tracking and Segmentation
- Authors: Alzayat Saleh, Marcus Sheaves, Dean Jerry, Mostafa Rahimi Azghadi
- Abstract summary: We propose a three-stage framework for robust fish tracking and segmentation.
The first stage is an optical flow model, which generates the pseudo labels using spatial and temporal consistency between frames.
In the second stage, a self-supervised model refines the pseudo-labels incrementally.
In the third stage, the refined labels are used to train a segmentation network.
- Score: 2.1028463367241033
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: DNN for fish tracking and segmentation based on high-quality labels is
expensive. Alternative unsupervised approaches rely on spatial and temporal
variations that naturally occur in video data to generate noisy
pseudo-ground-truth labels. These pseudo-labels are used to train a multi-task
deep neural network. In this paper, we propose a three-stage framework for
robust fish tracking and segmentation, where the first stage is an optical flow
model, which generates the pseudo labels using spatial and temporal consistency
between frames. In the second stage, a self-supervised model refines the
pseudo-labels incrementally. In the third stage, the refined labels are used to
train a segmentation network. No human annotations are used during the training
or inference. Extensive experiments are performed to validate our method on
three public underwater video datasets and to demonstrate that it is highly
effective for video annotation and segmentation. We also evaluate the
robustness of our framework to different imaging conditions and discuss the
limitations of our current implementation.
Related papers
- Label-Efficient 3D Brain Segmentation via Complementary 2D Diffusion Models with Orthogonal Views [10.944692719150071]
We propose a novel 3D brain segmentation approach using complementary 2D diffusion models.
Our goal is to achieve reliable segmentation quality without requiring complete labels for each individual subject.
arXiv Detail & Related papers (2024-07-17T06:14:53Z) - WildScenes: A Benchmark for 2D and 3D Semantic Segmentation in
Large-scale Natural Environments [34.24004079703609]
We introduce WildScenes, a bi-modal benchmark dataset consisting of multiple large-scales in natural environments.
The data is trajectory-centric with accurate localization and globally aligned point clouds.
We introduce benchmarks on 2D and 3D semantic segmentation and evaluate a variety of recent deep-learning techniques.
arXiv Detail & Related papers (2023-12-23T22:27:40Z) - Beyond the Label Itself: Latent Labels Enhance Semi-supervised Point
Cloud Panoptic Segmentation [46.01433705072047]
We find two types of latent labels behind the displayed label embedded in LiDAR and image data.
We propose a novel augmentation, Cylinder-Mix, which is able to augment more yet reliable samples for training.
We also propose the Instance Position-scale Learning (IPSL) Module to learn and fuse the information of instance position and scale.
arXiv Detail & Related papers (2023-12-13T15:56:24Z) - Self-Supervised 3D Scene Flow Estimation and Motion Prediction using
Local Rigidity Prior [100.98123802027847]
We investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds.
We generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation.
Our method achieves new state-of-the-art performance in self-supervised scene flow learning.
arXiv Detail & Related papers (2023-10-17T14:06:55Z) - LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds.
Our method co-designs an efficient labeling process with semi/weakly supervised learning.
Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
arXiv Detail & Related papers (2022-10-14T19:13:36Z) - Image Understands Point Cloud: Weakly Supervised 3D Semantic
Segmentation via Association Learning [59.64695628433855]
We propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images.
Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels.
Our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.
arXiv Detail & Related papers (2022-09-16T07:59:04Z) - Collaborative Propagation on Multiple Instance Graphs for 3D Instance
Segmentation with Single-point Supervision [63.429704654271475]
We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point.
With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information.
Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
arXiv Detail & Related papers (2022-08-10T02:14:39Z) - Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
We train a semantic point cloud segmentation network with only a small portion of points being labeled.
We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples.
Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
arXiv Detail & Related papers (2021-07-23T14:34:57Z) - One Thing One Click: A Self-Training Approach for Weakly Supervised 3D
Semantic Segmentation [78.36781565047656]
We propose "One Thing One Click," meaning that the annotator only needs to label one point per object.
We iteratively conduct the training and label propagation, facilitated by a graph propagation module.
Our results are also comparable to those of the fully supervised counterparts.
arXiv Detail & Related papers (2021-04-06T02:27:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.