Self-Supervised Multi-Object Tracking For Autonomous Driving From Consistency Across Timescales

• URL: http://arxiv.org/abs/2304.13147v2
• Date: Thu, 21 Sep 2023 12:07:07 GMT
• Title: Self-Supervised Multi-Object Tracking For Autonomous Driving From Consistency Across Timescales
• Authors: Christopher Lang, Alexander Braun, Lars Schillingmann, Abhinav Valada
• Abstract summary: Self-supervised multi-object trackers have tremendous potential as they enable learning from raw domain-specific data. However, their re-identification accuracy still falls short compared to their supervised counterparts. We propose a training objective that enables self-supervised learning of re-identification features from multiple sequential frames.
• Score: 53.55369862746357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised multi-object trackers have tremendous potential as they enable learning from raw domain-specific data. However, their re-identification accuracy still falls short compared to their supervised counterparts. We hypothesize that this drawback results from formulating self-supervised objectives that are limited to single frames or frame pairs. Such formulations do not capture sufficient visual appearance variations to facilitate learning consistent re-identification features for autonomous driving when the frame rate is low or object dynamics are high. In this work, we propose a training objective that enables self-supervised learning of re-identification features from multiple sequential frames by enforcing consistent association scores across short and long timescales. We perform extensive evaluations demonstrating that re-identification features trained from longer sequences significantly reduce ID switches on standard autonomous driving datasets compared to existing self-supervised learning methods, which are limited to training on frame pairs. Using our proposed SubCo loss function, we set the new state-of-the-art among self-supervised methods and even perform on par with fully supervised learning methods.

Related papers

• CORT: Class-Oriented Real-time Tracking for Embedded Systems [46.3107850275261]
  This work proposes a new approach to multi-class object tracking. It allows achieving smaller and more predictable execution times, without penalizing the tracking performance. The proposed solution was evaluated in complex urban scenarios with several objects of different types.
  arXiv  Detail & Related papers  (2024-07-20T09:12:17Z)
• Unsupervised 3D registration through optimization-guided cyclical self-training [71.75057371518093]
  State-of-the-art deep learning-based registration methods employ three different learning strategies. We propose a novel self-supervised learning paradigm for unsupervised registration, relying on self-training. We evaluate the method for abdomen and lung registration, consistently surpassing metric-based supervision and outperforming diverse state-of-the-art competitors.
  arXiv  Detail & Related papers  (2023-06-29T14:54:10Z)
• S$^3$Track: Self-supervised Tracking with Soft Assignment Flow [45.77333923477176]
  We study self-supervised multiple object tracking without using any video-level association labels. We propose differentiable soft object assignment for object association. We evaluate our proposed model on the KITTI, nuScenes, and Argoverse datasets.
  arXiv  Detail & Related papers  (2023-05-17T06:25:40Z)
• Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequences [49.91741677556553]
  We propose TempO, a temporal ordering pretext task for pre-training region-level feature representations for perception tasks. We embed each frame by an unordered set of proposal feature vectors, a representation that is natural for object detection or tracking systems. Extensive evaluations on the BDD100K, nuImages, and MOT17 datasets show that our TempO pre-training approach outperforms single-frame self-supervised learning methods.
  arXiv  Detail & Related papers  (2023-02-17T18:18:27Z)
• Self-Supervised Models are Continual Learners [79.70541692930108]
  We show that self-supervised loss functions can be seamlessly converted into distillation mechanisms for Continual Learning. We devise a framework for Continual self-supervised visual representation Learning that significantly improves the quality of the learned representations.
  arXiv  Detail & Related papers  (2021-12-08T10:39:13Z)
• Temporal Continuity Based Unsupervised Learning for Person Re-Identification [15.195514083289801]
  We propose an unsupervised center-based clustering approach capable of progressively learning and exploiting the underlying re-id discriminative information. We call our framework Temporal Continuity based Unsupervised Learning (TCUL) Specifically, TCUL simultaneously does center based clustering of unlabeled (target) dataset and fine-tunes a convolutional neural network (CNN) pre-trained on irrelevant labeled (source) dataset. It exploits temporally continuous nature of images within-camera jointly with spatial similarity of feature maps across-cameras to generate reliable pseudo-labels for training a re-identification model.
  arXiv  Detail & Related papers  (2020-09-01T05:29:30Z)
• CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions [61.724894233252414]
  This paper proposes a self-supervised learning method for the person re-identification (re-ID) problem. Existing unsupervised methods usually rely on pseudo labels, such as those from video tracklets or clustering. We introduce a different unsupervised method that allows us to learn pedestrian embeddings from raw videos, without resorting to pseudo labels.
  arXiv  Detail & Related papers  (2020-07-15T09:52:35Z)
• Self-supervised Video Object Segmentation [76.83567326586162]
  The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking) We make the following contributions: (i) we propose to improve the existing self-supervised approach, with a simple, yet more effective memory mechanism for long-term correspondence matching; (ii) by augmenting the self-supervised approach with an online adaptation module, our method successfully alleviates tracker drifts caused by spatial-temporal discontinuity; (iv) we demonstrate state-of-the-art results among the self-supervised approaches on DAVIS-2017 and YouTube
  arXiv  Detail & Related papers  (2020-06-22T17:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.