TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses

• URL: http://arxiv.org/abs/2306.05888v2
• Date: Fri, 18 Aug 2023 08:31:15 GMT
• Title: TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses
• Authors: Xuesong Chen, Shaoshuai Shi, Chao Zhang, Benjin Zhu, Qiang Wang, Ka Chun Cheung, Simon See, Hongsheng Li
• Abstract summary: 3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots. We present TrajectoryFormer, a novel point-cloud-based 3D MOT framework.
• Score: 51.60422927416087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots. With the commonly used tracking-by-detection paradigm, 3D MOT has made important progress in recent years. However, these methods only use the detection boxes of the current frame to obtain trajectory-box association results, which makes it impossible for the tracker to recover objects missed by the detector. In this paper, we present TrajectoryFormer, a novel point-cloud-based 3D MOT framework. To recover the missed object by detector, we generates multiple trajectory hypotheses with hybrid candidate boxes, including temporally predicted boxes and current-frame detection boxes, for trajectory-box association. The predicted boxes can propagate object's history trajectory information to the current frame and thus the network can tolerate short-term miss detection of the tracked objects. We combine long-term object motion feature and short-term object appearance feature to create per-hypothesis feature embedding, which reduces the computational overhead for spatial-temporal encoding. Additionally, we introduce a Global-Local Interaction Module to conduct information interaction among all hypotheses and models their spatial relations, leading to accurate estimation of hypotheses. Our TrajectoryFormer achieves state-of-the-art performance on the Waymo 3D MOT benchmarks. Code is available at https://github.com/poodarchu/EFG .

Related papers

• PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection [66.94819989912823]
  We propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection. We use point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement. We conduct extensive experiments on the large-scale dataset to demonstrate that our approach performs well against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-13T18:59:13Z)
• Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
  We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking. We propose MoMA-M3T, a framework that mainly consists of three motion-aware components. We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-22T17:53:58Z)
• A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds [50.54083964183614]
  It is non-trivial to perform accurate target-specific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. We propose DMT, a Detector-free Motion prediction based 3D Tracking network that totally removes the usage of complicated 3D detectors.
  arXiv  Detail & Related papers  (2022-03-08T17:49:07Z)
• 3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation [0.0]
  3D-FCT is a Siamese network architecture that utilizes temporal information to simultaneously perform the related tasks of 3D object detection and tracking. Our proposed method is evaluated on the KITTI tracking dataset where it is shown to provide an improvement of 5.57% mAP over a state-of-the-art approach.
  arXiv  Detail & Related papers  (2021-10-06T06:36:29Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking [34.40019455462043]
  We propose to learn 3D object correspondences from temporal point cloud data and infer the motion information from correspondence patterns. Our method exceeds the existing 3D tracking methods on both the KITTI and larger scale Nuscenes dataset.
  arXiv  Detail & Related papers  (2020-10-20T06:07:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.