MCTR: Multi Camera Tracking Transformer

• URL: http://arxiv.org/abs/2408.13243v2
• Date: Wed, 11 Sep 2024 16:41:15 GMT
• Title: MCTR: Multi Camera Tracking Transformer
• Authors: Alexandru Niculescu-Mizil, Deep Patel, Iain Melvin,
• Abstract summary: Multi-Camera Tracking tRansformer (MCTR) is a novel end-to-end approach tailored for multi-object detection and tracking across multiple cameras. MCTR leverages end-to-end detectors like DEtector TRansformer (DETR) to produce detections and detection embeddings independently for each camera view. The framework maintains set of track embeddings that encaplusate global information about the tracked objects, and updates them at every frame by integrating local information from the view-specific detection embeddings.
• Score: 45.66952089591361
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multi-camera tracking plays a pivotal role in various real-world applications. While end-to-end methods have gained significant interest in single-camera tracking, multi-camera tracking remains predominantly reliant on heuristic techniques. In response to this gap, this paper introduces Multi-Camera Tracking tRansformer (MCTR), a novel end-to-end approach tailored for multi-object detection and tracking across multiple cameras with overlapping fields of view. MCTR leverages end-to-end detectors like DEtector TRansformer (DETR) to produce detections and detection embeddings independently for each camera view. The framework maintains set of track embeddings that encaplusate global information about the tracked objects, and updates them at every frame by integrating the local information from the view-specific detection embeddings. The track embeddings are probabilistically associated with detections in every camera view and frame to generate consistent object tracks. The soft probabilistic association facilitates the design of differentiable losses that enable end-to-end training of the entire system. To validate our approach, we conduct experiments on MMPTrack and AI City Challenge, two recently introduced large-scale multi-camera multi-object tracking datasets.

Related papers

• RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Framework [28.359633046753228]
  We propose RockTrack, a 3D MOT method for multi-camera detectors. RockTrack incorporates a confidence-guided preprocessing module to extract reliable motion and image observations. RockTrack achieves state-of-the-art performance on the nuScenes vision-only tracking leaderboard with 59.1% AMOTA.
  arXiv  Detail & Related papers  (2024-09-18T07:08:08Z)
• MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark [63.878793340338035]
  Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. Existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting. We present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments.
  arXiv  Detail & Related papers  (2024-03-29T15:08:37Z)
• Learning Online Policies for Person Tracking in Multi-View Environments [4.62316736194615]
  We introduce MVSparse, a novel framework for cooperative multi-person tracking across multiple synchronized cameras. The MVSparse system is comprised of a carefully orchestrated pipeline, combining edge server-based models with distributed lightweight Reinforcement Learning (RL) agents. Notably, our contributions include an empirical analysis of multi-camera pedestrian tracking datasets, the development of a multi-camera, multi-person detection pipeline, and the implementation of MVSparse.
  arXiv  Detail & Related papers  (2023-12-26T02:57:11Z)
• Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking [55.13878429987136]
  We propose a simple yet effective two-stage feature learning paradigm to jointly learn single-shot and multi-shot features for different targets. Our method has achieved significant improvements on MOT17 and MOT20 datasets while reaching state-of-the-art performance on DanceTrack dataset.
  arXiv  Detail & Related papers  (2023-11-17T08:17:49Z)
• DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes [74.64897845999677]
  We introduce a new cross-view multi-object tracking dataset for DIVerse Open scenes with dense tracking pedestrians. Our DIVOTrack has fifteen distinct scenarios and 953 cross-view tracks, surpassing all cross-view multi-object tracking datasets currently available. Furthermore, we provide a novel baseline cross-view tracking method with a unified joint detection and cross-view tracking framework named CrossMOT.
  arXiv  Detail & Related papers  (2023-02-15T14:10:42Z)
• Multi-Camera Multi-Object Tracking on the Move via Single-Stage Global Association Approach [23.960847268459293]
  This work introduces novel Single-Stage Global Association Tracking approaches to associate one or more detection from multi-cameras with tracked objects. Our models also improve the detection accuracy of the standard vision-based 3D object detectors in the nuScenes detection challenge.
  arXiv  Detail & Related papers  (2022-11-17T17:03:24Z)
• Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration [56.01625477187448]
  We propose a MultiModality PAnoramic multi-object Tracking framework (MMPAT) It takes both 2D panorama images and 3D point clouds as input and then infers target trajectories using the multimodality data. We evaluate the proposed method on the JRDB dataset, where the MMPAT achieves the top performance in both the detection and tracking tasks.
  arXiv  Detail & Related papers  (2021-05-31T03:16:38Z)
• TrackFormer: Multi-Object Tracking with Transformers [92.25832593088421]
  TrackFormer is an end-to-end multi-object tracking and segmentation model based on an encoder-decoder Transformer architecture. New track queries are spawned by the DETR object detector and embed the position of their corresponding object over time. TrackFormer achieves a seamless data association between frames in a new tracking-by-attention paradigm.
  arXiv  Detail & Related papers  (2021-01-07T18:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.