Uncertainty-aware Self-supervised 3D Data Association

• URL: http://arxiv.org/abs/2008.08173v1
• Date: Tue, 18 Aug 2020 22:34:07 GMT
• Title: Uncertainty-aware Self-supervised 3D Data Association
• Abstract summary: We propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers. We show how these self-supervised annotations can be used in a principled manner to learn point-cloud embeddings. We design embeddings to differentiate objects across frames, and learn them using uncertainty-aware self-supervised training.
• Score: 20.853544785595343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D object trackers usually require training on large amounts of annotated data that is expensive and time-consuming to collect. Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association. Large scale annotations for unlabeled data are cheaply obtained by automatic object detection and association across frames. We show how these self-supervised annotations can be used in a principled manner to learn point-cloud embeddings that are effective for 3D tracking. We estimate and incorporate uncertainty in self-supervised tracking to learn more robust embeddings, without needing any labeled data. We design embeddings to differentiate objects across frames, and learn them using uncertainty-aware self-supervised training. Finally, we demonstrate their ability to perform accurate data association across frames, towards effective and accurate 3D tracking. Project videos and code are at https://jianrenw.github.io/Self-Supervised-3D-Data-Association.

Related papers

• Lifting Unlabeled Internet-level Data for 3D Scene Understanding [56.35787129098819]
  We demonstrate that carefully designed data engines can leverage web-curated, unlabeled videos to automatically generate training data.<n>We identify and analyze bottlenecks in automated data generation, revealing critical factors that determine the efficiency and effectiveness of learning from unlabeled data.
  arXiv  Detail & Related papers  (2026-04-02T11:26:44Z)
• Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detection [52.66283064389691]
  State-of-the-art 3D object detectors are often trained on massive labeled datasets. Recent works demonstrate that self-supervised pre-training with unlabeled data can improve detection accuracy with limited labels. We propose a shelf-supervised approach for generating zero-shot 3D bounding boxes from paired RGB and LiDAR data.
  arXiv  Detail & Related papers  (2024-06-14T15:21:57Z)
• SeMoLi: What Moves Together Belongs Together [51.72754014130369]
  We tackle semi-supervised object detection based on motion cues. Recent results suggest that motion-based clustering methods can be used to pseudo-label instances of moving objects. We re-think this approach and suggest that both, object detection, as well as motion-inspired pseudo-labeling, can be tackled in a data-driven manner.
  arXiv  Detail & Related papers  (2024-02-29T18:54:53Z)
• View-to-Label: Multi-View Consistency for Self-Supervised 3D Object Detection [46.077668660248534]
  We propose a novel approach to self-supervise 3D object detection purely from RGB sequences alone. Our experiments on KITTI 3D dataset demonstrate performance on par with state-of-the-art self-supervised methods.
  arXiv  Detail & Related papers  (2023-05-29T09:30:39Z)
• S$^2$Contact: Graph-based Network for 3D Hand-Object Contact Estimation with Semi-Supervised Learning [70.72037296392642]
  We propose a novel semi-supervised framework that allows us to learn contact from monocular images. Specifically, we leverage visual and geometric consistency constraints in large-scale datasets for generating pseudo-labels. We show benefits from using a contact map that rules hand-object interactions to produce more accurate reconstructions.
  arXiv  Detail & Related papers  (2022-08-01T14:05:23Z)
• 3D Object Detection with a Self-supervised Lidar Scene Flow Backbone [10.341296683155973]
  We propose using a self-supervised training strategy to learn a general point cloud backbone model for downstream 3D vision tasks. Our main contribution leverages learned flow and motion representations and combines a self-supervised backbone with a 3D detection head. Experiments on KITTI and nuScenes benchmarks show that the proposed self-supervised pre-training increases 3D detection performance significantly.
  arXiv  Detail & Related papers  (2022-05-02T07:53:29Z)
• Semi-supervised 3D Object Detection via Adaptive Pseudo-Labeling [18.209409027211404]
  3D object detection is an important task in computer vision. Most existing methods require a large number of high-quality 3D annotations, which are expensive to collect. We propose a novel semi-supervised framework based on pseudo-labeling for outdoor 3D object detection tasks.
  arXiv  Detail & Related papers  (2021-08-15T02:58:43Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking [34.40019455462043]
  We propose to learn 3D object correspondences from temporal point cloud data and infer the motion information from correspondence patterns. Our method exceeds the existing 3D tracking methods on both the KITTI and larger scale Nuscenes dataset.
  arXiv  Detail & Related papers  (2020-10-20T06:07:20Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.