Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

• URL: http://arxiv.org/abs/2302.03802v2
• Date: Mon, 3 Apr 2023 14:49:49 GMT
• Title: Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking
• Authors: Ziqi Pang, Jie Li, Pavel Tokmakov, Dian Chen, Sergey Zagoruyko, Yu-Xiong Wang
• Abstract summary: We propose an end-to-end multi-camera 3D multi-object tracking framework. We name it "Past-and-Future reasoning for Tracking" (PFTrack)
• Score: 30.357116118917368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work proposes an end-to-end multi-camera 3D multi-object tracking (MOT) framework. It emphasizes spatio-temporal continuity and integrates both past and future reasoning for tracked objects. Thus, we name it "Past-and-Future reasoning for Tracking" (PF-Track). Specifically, our method adapts the "tracking by attention" framework and represents tracked instances coherently over time with object queries. To explicitly use historical cues, our "Past Reasoning" module learns to refine the tracks and enhance the object features by cross-attending to queries from previous frames and other objects. The "Future Reasoning" module digests historical information and predicts robust future trajectories. In the case of long-term occlusions, our method maintains the object positions and enables re-association by integrating motion predictions. On the nuScenes dataset, our method improves AMOTA by a large margin and remarkably reduces ID-Switches by 90% compared to prior approaches, which is an order of magnitude less. The code and models are made available at https://github.com/TRI-ML/PF-Track.

Related papers

• PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection [66.94819989912823]
  We propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection. We use point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement. We conduct extensive experiments on the large-scale dataset to demonstrate that our approach performs well against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-13T18:59:13Z)
• TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses [51.60422927416087]
  3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots. We present TrajectoryFormer, a novel point-cloud-based 3D MOT framework.
  arXiv  Detail & Related papers  (2023-06-09T13:31:50Z)
• Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection [50.959453059206446]
  This paper aims for high-performance offline LiDAR-based 3D object detection. We first observe that experienced human annotators annotate objects from a track-centric perspective. We propose a high-performance offline detector in a track-centric perspective instead of the conventional object-centric perspective.
  arXiv  Detail & Related papers  (2023-04-24T17:59:05Z)
• SpOT: Spatiotemporal Modeling for 3D Object Tracking [68.12017780034044]
  3D multi-object tracking aims to consistently identify all mobile time. Current 3D tracking methods rely on abstracted information and limited history. We develop a holistic representation of scenes that leverage both spatial and temporal information.
  arXiv  Detail & Related papers  (2022-07-12T21:45:49Z)
• Multi-Object Tracking and Segmentation with a Space-Time Memory Network [12.043574473965318]
  We propose a method for multi-object tracking and segmentation based on a novel memory-based mechanism to associate tracklets. The proposed tracker, MeNToS, addresses particularly the long-term data association problem.
  arXiv  Detail & Related papers  (2021-10-21T17:13:17Z)
• MOTR: End-to-End Multiple-Object Tracking with TRansformer [31.78906135775541]
  We present MOTR, the first fully end-to-end multiple object tracking framework. It learns to model the long-range temporal variation of the objects. Results show that MOTR achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-07T13:27:01Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving [22.693895321632507]
  We propose a probabilistic, multi-modal, multi-object tracking system consisting of different trainable modules. We show that our method outperforms current state-of-the-art on the NuScenes Tracking dataset.
  arXiv  Detail & Related papers  (2020-12-26T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.