Real-time 3D Deep Multi-Camera Tracking
- URL: http://arxiv.org/abs/2003.11753v1
- Date: Thu, 26 Mar 2020 06:08:19 GMT
- Title: Real-time 3D Deep Multi-Camera Tracking
- Authors: Quanzeng You, Hao Jiang
- Abstract summary: We propose a novel end-to-end tracking pipeline, Deep Multi-Camera Tracking (DMCT), which achieves reliable real-time multi-camera people tracking.
Our system achieves the state-of-the-art tracking results while maintaining real-time performance.
- Score: 13.494550690138775
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Tracking a crowd in 3D using multiple RGB cameras is a challenging task. Most
previous multi-camera tracking algorithms are designed for offline setting and
have high computational complexity. Robust real-time multi-camera 3D tracking
is still an unsolved problem. In this work, we propose a novel end-to-end
tracking pipeline, Deep Multi-Camera Tracking (DMCT), which achieves reliable
real-time multi-camera people tracking. Our DMCT consists of 1) a fast and
novel perspective-aware Deep GroudPoint Network, 2) a fusion procedure for
ground-plane occupancy heatmap estimation, 3) a novel Deep Glimpse Network for
person detection and 4) a fast and accurate online tracker. Our design fully
unleashes the power of deep neural network to estimate the "ground point" of
each person in each color image, which can be optimized to run efficiently and
robustly. Our fusion procedure, glimpse network and tracker merge the results
from different views, find people candidates using multiple video frames and
then track people on the fused heatmap. Our system achieves the
state-of-the-art tracking results while maintaining real-time performance.
Apart from evaluation on the challenging WILDTRACK dataset, we also collect two
more tracking datasets with high-quality labels from two different environments
and camera settings. Our experimental results confirm that our proposed
real-time pipeline gives superior results to previous approaches.
Related papers
- Lifting Multi-View Detection and Tracking to the Bird's Eye View [5.679775668038154]
Recent advancements in multi-view detection and 3D object recognition have significantly improved performance.
We compare modern lifting methods, both parameter-free and parameterized, to multi-view aggregation.
We present an architecture that aggregates the features of multiple times steps to learn robust detection.
arXiv Detail & Related papers (2024-03-19T09:33:07Z) - ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every
Detection Box [81.45219802386444]
Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects across video frames.
We propose a hierarchical data association strategy to mine the true objects in low-score detection boxes.
In 3D scenarios, it is much easier for the tracker to predict object velocities in the world coordinate.
arXiv Detail & Related papers (2023-03-27T15:35:21Z) - Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object
Detection and Tracking [53.64390261936975]
We present Minkowski Tracker, a sparse-temporal R-CNN that jointly solves object detection and tracking problems.
Inspired by region-based CNN (R-CNN), we propose to track motion as a second stage of the object detector R-CNN.
We show in large-scale experiments that the overall performance gain of our method is due to four factors.
arXiv Detail & Related papers (2022-08-22T04:47:40Z) - Scalable and Real-time Multi-Camera Vehicle Detection,
Re-Identification, and Tracking [58.95210121654722]
We propose a real-time city-scale multi-camera vehicle tracking system that handles real-world, low-resolution CCTV instead of idealized and curated video streams.
Our method is ranked among the top five performers on the public leaderboard.
arXiv Detail & Related papers (2022-04-15T12:47:01Z) - MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People
Tracking Benchmark [40.363608495563305]
We provide a large-scale densely-labeled multi-camera tracking dataset in five different environments with the help of an auto-annotation system.
The 3D tracking results are projected to each RGB camera view using camera parameters to create 2D tracking results.
This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments.
arXiv Detail & Related papers (2021-11-30T06:29:14Z) - LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera
Multi-Object Tracking [42.87953709286856]
Multi-Camera Multi-Object Tracking is currently drawing attention in the computer vision field due to its superior performance in real-world applications.
We propose a mathematically elegant multi-camera multiple object tracking approach based on a spatial-temporal lifted multicut formulation.
arXiv Detail & Related papers (2021-11-23T14:09:47Z) - CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object
Tracking [9.62721286522053]
We propose an end-to-end network for joint object detection and tracking based on radar and camera sensor fusion.
Our proposed method uses a center-based radar-camera fusion algorithm for object detection and utilizes a greedy algorithm for object association.
We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark.
arXiv Detail & Related papers (2021-07-11T23:56:53Z) - Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving.
We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
arXiv Detail & Related papers (2021-03-12T15:30:02Z) - Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion
Forecasting with a Single Convolutional Net [93.51773847125014]
We propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor.
Our approach performs 3D convolutions across space and time over a bird's eye view representation of the 3D world.
arXiv Detail & Related papers (2020-12-22T22:43:35Z) - Tracking-by-Counting: Using Network Flows on Crowd Density Maps for
Tracking Multiple Targets [96.98888948518815]
State-of-the-art multi-object tracking(MOT) methods follow the tracking-by-detection paradigm.
We propose a new MOT paradigm, tracking-by-counting, tailored for crowded scenes.
arXiv Detail & Related papers (2020-07-18T19:51:53Z) - JRMOT: A Real-Time 3D Multi-Object Tracker and a New Large-Scale Dataset [34.609125601292]
We present JRMOT, a novel 3D MOT system that integrates information from RGB images and 3D point clouds to achieve real-time tracking performance.
As part of our work, we release the JRDB dataset, a novel large scale 2D+3D dataset and benchmark.
The presented 3D MOT system demonstrates state-of-the-art performance against competing methods on the popular 2D tracking KITTI benchmark.
arXiv Detail & Related papers (2020-02-19T19:21:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.