3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention

• URL: http://arxiv.org/abs/2203.10926v1
• Date: Mon, 21 Mar 2022 12:44:17 GMT
• Title: 3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention
• Authors: Martin Buchner and Abhinav Valada
• Abstract summary: Batch3DMOT represents real-world scenes as directed, acyclic, and category-disjoint tracking graphs. We present a multi-modal graph neural network that uses a cross-edge attention mechanism mitigating modality intermittence.
• Score: 9.150245363036165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online 3D multi-object tracking (MOT) has witnessed significant research interest in recent years, largely driven by demand from the autonomous systems community. However, 3D offline MOT is relatively less explored. Labeling 3D trajectory scene data at a large scale while not relying on high-cost human experts is still an open research question. In this work, we propose Batch3DMOT that follows the tracking-by-detection paradigm and represents real-world scenes as directed, acyclic, and category-disjoint tracking graphs that are attributed using various modalities such as camera, LiDAR, and radar. We present a multi-modal graph neural network that uses a cross-edge attention mechanism mitigating modality intermittence, which translates into sparsity in the graph domain. Additionally, we present attention-weighted convolutions over frame-wise k-NN neighborhoods as suitable means to allow information exchange across disconnected graph components. We evaluate our approach using various sensor modalities and model configurations on the challenging nuScenes and KITTI datasets. Extensive experiments demonstrate that our proposed approach yields an overall improvement of 2.8% in the AMOTA score on nuScenes thereby setting a new benchmark for 3D tracking methods and successfully enhances false positive filtering.

Related papers

• OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagation [67.56268991234371]
  OV-Uni3DETR achieves the state-of-the-art performance on various scenarios, surpassing existing methods by more than 6% on average. Code and pre-trained models will be released later.
  arXiv  Detail & Related papers  (2024-03-28T17:05:04Z)
• 3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking [15.330384668966806]
  State-of-the-art 3D multi-object tracking (MOT) approaches typically rely on non-learned model-based algorithms such as Kalman Filter. We propose 3DMOTFormer, a learned geometry-based 3D MOT framework building upon the transformer architecture. Our approach achieves 71.2% and 68.2% AMOTA on the nuScenes validation and test split, respectively.
  arXiv  Detail & Related papers  (2023-08-12T19:19:58Z)
• Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
  This paper presents a novel approach that views each tracklet as a continuous stream. At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank. To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
  arXiv  Detail & Related papers  (2023-03-14T02:58:27Z)
• Unleash the Potential of Image Branch for Cross-modal 3D Object Detection [67.94357336206136]
  We present a new cross-modal 3D object detector, namely UPIDet, which aims to unleash the potential of the image branch from two aspects. First, UPIDet introduces a new 2D auxiliary task called normalized local coordinate map estimation. Second, we discover that the representational capability of the point cloud backbone can be enhanced through the gradients backpropagated from the training objectives of the image branch.
  arXiv  Detail & Related papers  (2023-01-22T08:26:58Z)
• 3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds [95.54285993019843]
  We propose a method for joint detection and tracking of multiple objects in 3D point clouds. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network.
  arXiv  Detail & Related papers  (2022-11-01T20:59:38Z)
• InterTrack: Interaction Transformer for 3D Multi-Object Tracking [9.283656931246645]
  3D multi-object tracking (MOT) is a key problem for autonomous vehicles. Our proposed solution, InterTrack, generates discriminative object representations for data association. We validate our approach on the nuScenes 3D MOT benchmark, where we observe significant improvements.
  arXiv  Detail & Related papers  (2022-08-17T03:24:36Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Graph Neural Networks for 3D Multi-Object Tracking [28.121708602059048]
  3D Multi-object tracking (MOT) is crucial to autonomous systems. Recent work often uses a tracking-by-detection pipeline. We propose a novel feature interaction mechanism by introducing Graph Neural Networks.
  arXiv  Detail & Related papers  (2020-08-20T17:55:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.