RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Framework

• URL: http://arxiv.org/abs/2409.11749v1
• Date: Wed, 18 Sep 2024 07:08:08 GMT
• Title: RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Framework
• Authors: Xiaoyu Li, Peidong Li, Lijun Zhao, Dedong Liu, Jinghan Gao, Xian Wu, Yitao Wu, Dixiao Cui,
• Abstract summary: We propose RockTrack, a 3D MOT method for multi-camera detectors. RockTrack incorporates a confidence-guided preprocessing module to extract reliable motion and image observations. RockTrack achieves state-of-the-art performance on the nuScenes vision-only tracking leaderboard with 59.1% AMOTA.
• Score: 28.359633046753228
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: 3D Multi-Object Tracking (MOT) obtains significant performance improvements with the rapid advancements in 3D object detection, particularly in cost-effective multi-camera setups. However, the prevalent end-to-end training approach for multi-camera trackers results in detector-specific models, limiting their versatility. Moreover, current generic trackers overlook the unique features of multi-camera detectors, i.e., the unreliability of motion observations and the feasibility of visual information. To address these challenges, we propose RockTrack, a 3D MOT method for multi-camera detectors. Following the Tracking-By-Detection framework, RockTrack is compatible with various off-the-shelf detectors. RockTrack incorporates a confidence-guided preprocessing module to extract reliable motion and image observations from distinct representation spaces from a single detector. These observations are then fused in an association module that leverages geometric and appearance cues to minimize mismatches. The resulting matches are propagated through a staged estimation process, forming the basis for heuristic noise modeling. Additionally, we introduce a novel appearance similarity metric for explicitly characterizing object affinities in multi-camera settings. RockTrack achieves state-of-the-art performance on the nuScenes vision-only tracking leaderboard with 59.1% AMOTA while demonstrating impressive computational efficiency.

Related papers

• MCTR: Multi Camera Tracking Transformer [45.66952089591361]
  Multi-Camera Tracking tRansformer (MCTR) is a novel end-to-end approach tailored for multi-object detection and tracking across multiple cameras. MCTR leverages end-to-end detectors like DEtector TRansformer (DETR) to produce detections and detection embeddings independently for each camera view. The framework maintains set of track embeddings that encaplusate global information about the tracked objects, and updates them at every frame by integrating local information from the view-specific detection embeddings.
  arXiv  Detail & Related papers  (2024-08-23T17:37:03Z)
• Lifting Multi-View Detection and Tracking to the Bird's Eye View [5.679775668038154]
  Recent advancements in multi-view detection and 3D object recognition have significantly improved performance. We compare modern lifting methods, both parameter-free and parameterized, to multi-view aggregation. We present an architecture that aggregates the features of multiple times steps to learn robust detection.
  arXiv  Detail & Related papers  (2024-03-19T09:33:07Z)
• Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking [55.13878429987136]
  We propose a simple yet effective two-stage feature learning paradigm to jointly learn single-shot and multi-shot features for different targets. Our method has achieved significant improvements on MOT17 and MOT20 datasets while reaching state-of-the-art performance on DanceTrack dataset.
  arXiv  Detail & Related papers  (2023-11-17T08:17:49Z)
• Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
  We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking. We propose MoMA-M3T, a framework that mainly consists of three motion-aware components. We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-22T17:53:58Z)
• ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box [81.45219802386444]
  Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects across video frames. We propose a hierarchical data association strategy to mine the true objects in low-score detection boxes. In 3D scenarios, it is much easier for the tracker to predict object velocities in the world coordinate.
  arXiv  Detail & Related papers  (2023-03-27T15:35:21Z)
• A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds [50.54083964183614]
  It is non-trivial to perform accurate target-specific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. We propose DMT, a Detector-free Motion prediction based 3D Tracking network that totally removes the usage of complicated 3D detectors.
  arXiv  Detail & Related papers  (2022-03-08T17:49:07Z)
• MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark [40.363608495563305]
  We provide a large-scale densely-labeled multi-camera tracking dataset in five different environments with the help of an auto-annotation system. The 3D tracking results are projected to each RGB camera view using camera parameters to create 2D tracking results. This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments.
  arXiv  Detail & Related papers  (2021-11-30T06:29:14Z)
• LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera Multi-Object Tracking [42.87953709286856]
  Multi-Camera Multi-Object Tracking is currently drawing attention in the computer vision field due to its superior performance in real-world applications. We propose a mathematically elegant multi-camera multiple object tracking approach based on a spatial-temporal lifted multicut formulation.
  arXiv  Detail & Related papers  (2021-11-23T14:09:47Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving [22.693895321632507]
  We propose a probabilistic, multi-modal, multi-object tracking system consisting of different trainable modules. We show that our method outperforms current state-of-the-art on the NuScenes Tracking dataset.
  arXiv  Detail & Related papers  (2020-12-26T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.