Spatiotemporal Self-supervised Learning for Point Clouds in the Wild
- URL: http://arxiv.org/abs/2303.16235v1
- Date: Tue, 28 Mar 2023 18:06:22 GMT
- Title: Spatiotemporal Self-supervised Learning for Point Clouds in the Wild
- Authors: Yanhao Wu, Tong Zhang, Wei Ke, Sabine S\"usstrunk, Mathieu Salzmann
- Abstract summary: We introduce an SSL strategy that leverages positive pairs in both the spatial and temporal domain.
We demonstrate the benefits of our approach via extensive experiments performed by self-supervised training on two large-scale LiDAR datasets.
- Score: 65.56679416475943
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Self-supervised learning (SSL) has the potential to benefit many
applications, particularly those where manually annotating data is cumbersome.
One such situation is the semantic segmentation of point clouds. In this
context, existing methods employ contrastive learning strategies and define
positive pairs by performing various augmentation of point clusters in a single
frame. As such, these methods do not exploit the temporal nature of LiDAR data.
In this paper, we introduce an SSL strategy that leverages positive pairs in
both the spatial and temporal domain. To this end, we design (i) a
point-to-cluster learning strategy that aggregates spatial information to
distinguish objects; and (ii) a cluster-to-cluster learning strategy based on
unsupervised object tracking that exploits temporal correspondences. We
demonstrate the benefits of our approach via extensive experiments performed by
self-supervised training on two large-scale LiDAR datasets and transferring the
resulting models to other point cloud segmentation benchmarks. Our results
evidence that our method outperforms the state-of-the-art point cloud SSL
methods.
Related papers
- Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange [50.45953583802282]
We introduce a novel self-supervised learning (SSL) strategy for point cloud scene understanding.
Our approach leverages both object patterns and contextual cues to produce robust features.
Our experiments demonstrate the superiority of our method over existing SSL techniques.
arXiv Detail & Related papers (2024-04-11T06:39:53Z) - A Spatiotemporal Correspondence Approach to Unsupervised LiDAR
Segmentation with Traffic Applications [16.260518238832887]
Key idea is to leverage the nature of a dynamic point cloud sequence and introduce drastically stronger scenarios.
We alternate between optimizing semantic into groups and clustering using point-wisetemporal labels.
Our method can learn discriminative features in an unsupervised learning fashion.
arXiv Detail & Related papers (2023-08-23T21:32:46Z) - Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects [84.6945070729684]
Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks.
This article reviews current state-of-the-art SSL methods for time series data.
arXiv Detail & Related papers (2023-06-16T18:23:10Z) - Self-Supervised Learning for Point Clouds Data: A Survey [8.858165912687923]
Self-Supervised Learning (SSL) is considered as an essential solution to solve the time-consuming and labor-intensive data labelling problems.
This paper provides a comprehensive survey of recent advances on SSL for point clouds.
arXiv Detail & Related papers (2023-05-09T08:47:09Z) - De-coupling and De-positioning Dense Self-supervised Learning [65.56679416475943]
Dense Self-Supervised Learning (SSL) methods address the limitations of using image-level feature representations when handling images with multiple objects.
We show that they suffer from coupling and positional bias, which arise from the receptive field increasing with layer depth and zero-padding.
We demonstrate the benefits of our method on COCO and on a new challenging benchmark, OpenImage-MINI, for object classification, semantic segmentation, and object detection.
arXiv Detail & Related papers (2023-03-29T18:07:25Z) - PointCLM: A Contrastive Learning-based Framework for Multi-instance
Point Cloud Registration [4.969636478156443]
PointCLM is a contrastive learning-based framework for mutli-instance point cloud registration.
Our method outperforms the state-of-the-art methods on both synthetic and real datasets by a large margin.
arXiv Detail & Related papers (2022-09-01T04:30:05Z) - Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains.
We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
arXiv Detail & Related papers (2021-12-12T06:11:16Z) - On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks.
Despite these advances, the understanding of these methods is still relatively limited.
arXiv Detail & Related papers (2021-01-18T10:12:31Z) - Aggregative Self-Supervised Feature Learning from a Limited Sample [12.555160911451688]
We propose two strategies of aggregation in terms of complementarity of various forms to boost the robustness of self-supervised learned features.
Our experiments on 2D natural image and 3D medical image classification tasks under limited data scenarios confirm that the proposed aggregation strategies successfully boost the classification accuracy.
arXiv Detail & Related papers (2020-12-14T12:49:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.