Learning a Neural Association Network for Self-supervised Multi-Object Tracking

• URL: http://arxiv.org/abs/2411.11514v1
• Date: Mon, 18 Nov 2024 12:22:29 GMT
• Title: Learning a Neural Association Network for Self-supervised Multi-Object Tracking
• Authors: Shuai Li, Michael Burke, Subramanian Ramamoorthy, Juergen Gall,
• Abstract summary: This paper introduces a novel framework to learn data association for multi-object tracking in a self-supervised manner. Motivated by the fact that in real-world scenarios object motion can be usually represented by a Markov process, we present a novel expectation (EM) algorithm that trains a neural network to associate detections for tracking. We evaluate our approach on the challenging MOT17 and MOT20 datasets and achieve state-of-the-art results in comparison to self-supervised trackers.
• Score: 34.07776597698471
• License:
• Abstract: This paper introduces a novel framework to learn data association for multi-object tracking in a self-supervised manner. Fully-supervised learning methods are known to achieve excellent tracking performances, but acquiring identity-level annotations is tedious and time-consuming. Motivated by the fact that in real-world scenarios object motion can be usually represented by a Markov process, we present a novel expectation maximization (EM) algorithm that trains a neural network to associate detections for tracking, without requiring prior knowledge of their temporal correspondences. At the core of our method lies a neural Kalman filter, with an observation model conditioned on associations of detections parameterized by a neural network. Given a batch of frames as input, data associations between detections from adjacent frames are predicted by a neural network followed by a Sinkhorn normalization that determines the assignment probabilities of detections to states. Kalman smoothing is then used to obtain the marginal probability of observations given the inferred states, producing a training objective to maximize this marginal probability using gradient descent. The proposed framework is fully differentiable, allowing the underlying neural model to be trained end-to-end. We evaluate our approach on the challenging MOT17 and MOT20 datasets and achieve state-of-the-art results in comparison to self-supervised trackers using public detections. We furthermore demonstrate the capability of the learned model to generalize across datasets.

Related papers

• Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequences [49.91741677556553]
  We propose TempO, a temporal ordering pretext task for pre-training region-level feature representations for perception tasks. We embed each frame by an unordered set of proposal feature vectors, a representation that is natural for object detection or tracking systems. Extensive evaluations on the BDD100K, nuImages, and MOT17 datasets show that our TempO pre-training approach outperforms single-frame self-supervised learning methods.
  arXiv  Detail & Related papers  (2023-02-17T18:18:27Z)
• A Bayesian Detect to Track System for Robust Visual Object Tracking and Semi-Supervised Model Learning [1.7268829007643391]
  We ad-dress problems in a Bayesian tracking and detection framework parameterized by neural network outputs. We propose a particle filter-based approximate sampling algorithm for tracking object state estimation. Based on our particle filter inference algorithm, a semi-supervised learn-ing algorithm is utilized for learning tracking network on intermittent labeled frames.
  arXiv  Detail & Related papers  (2022-05-05T00:18:57Z)
• Neural Maximum A Posteriori Estimation on Unpaired Data for Motion Deblurring [87.97330195531029]
  We propose a Neural Maximum A Posteriori (NeurMAP) estimation framework for training neural networks to recover blind motion information and sharp content from unpaired data. The proposed NeurMAP is an approach to existing deblurring neural networks, and is the first framework that enables training image deblurring networks on unpaired datasets.
  arXiv  Detail & Related papers  (2022-04-26T08:09:47Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Learning data association without data association: An EM approach to neural assignment prediction [12.970250708769708]
  This paper introduces an expectation maximisation approach to train neural models for data association. It does not require labelling information to train a model for object recognition. Importantly, networks trained using the proposed approach can be re-used in downstream tracking applications.
  arXiv  Detail & Related papers  (2021-05-02T01:11:09Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition [77.77786072373942]
  This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition. We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space. Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
  arXiv  Detail & Related papers  (2021-02-09T03:07:07Z)
• TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking [8.791710193028903]
  This study follows many previous approaches to multi-object tracking (MOT) that model the problem using graph-based data structures. We create a framework based on dynamic undirected graphs that represent the data association problem over multiple timesteps. We also provide solutions and propositions for the computational problems that need to be addressed to create a memory-efficient, real-time, online algorithm.
  arXiv  Detail & Related papers  (2021-01-11T21:52:25Z)
• Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates [11.580916951856256]
  We present a time-dynamic approach to model uncertainties of instance segmentation networks. We apply this approach to the detection of false positives and the estimation of prediction quality. The proposed method only requires a readily trained neural network and video sequence input.
  arXiv  Detail & Related papers  (2020-12-14T13:39:05Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.