A Unified Multi-view Multi-person Tracking Framework
- URL: http://arxiv.org/abs/2302.03820v1
- Date: Wed, 8 Feb 2023 01:08:02 GMT
- Title: A Unified Multi-view Multi-person Tracking Framework
- Authors: Fan Yang, Shigeyuki Odashima, Sosuke Yamao, Hiroaki Fujimoto, Shoichi
Masui, and Shan Jiang
- Abstract summary: This study presents a Unified Multi-view Multi-person Tracking framework to bridge the gap between footprint tracking and pose tracking.
Without additional modifications, the framework can adopt monocular 2D bounding boxes and 2D poses as the input to produce robust 3D trajectories for multiple persons.
- Score: 5.143965432709092
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Although there is a significant development in 3D Multi-view Multi-person
Tracking (3D MM-Tracking), current 3D MM-Tracking frameworks are designed
separately for footprint and pose tracking. Specifically, frameworks designed
for footprint tracking cannot be utilized in 3D pose tracking, because they
directly obtain 3D positions on the ground plane with a homography projection,
which is inapplicable to 3D poses above the ground. In contrast, frameworks
designed for pose tracking generally isolate multi-view and multi-frame
associations and may not be robust to footprint tracking, since footprint
tracking utilizes fewer key points than pose tracking, which weakens multi-view
association cues in a single frame. This study presents a Unified Multi-view
Multi-person Tracking framework to bridge the gap between footprint tracking
and pose tracking. Without additional modifications, the framework can adopt
monocular 2D bounding boxes and 2D poses as the input to produce robust 3D
trajectories for multiple persons. Importantly, multi-frame and multi-view
information are jointly employed to improve the performance of association and
triangulation. The effectiveness of our framework is verified by accomplishing
state-of-the-art performance on the Campus and Shelf datasets for 3D pose
tracking, and by comparable results on the WILDTRACK and MMPTRACK datasets for
3D footprint tracking.
Related papers
- TAPVid-3D: A Benchmark for Tracking Any Point in 3D [63.060421798990845]
We introduce a new benchmark, TAPVid-3D, for evaluating the task of Tracking Any Point in 3D.
This benchmark will serve as a guidepost to improve our ability to understand precise 3D motion and surface deformation from monocular video.
arXiv Detail & Related papers (2024-07-08T13:28:47Z) - BiTrack: Bidirectional Offline 3D Multi-Object Tracking Using Camera-LiDAR Data [11.17376076195671]
"BiTrack" is a 3D OMOT framework that includes modules of 2D-3D detection fusion, initial trajectory generation, and bidirectional trajectory re-optimization.
The experiment results on the KITTI dataset demonstrate that BiTrack achieves the state-of-the-art performance for 3D OMOT tasks in terms of accuracy and efficiency.
arXiv Detail & Related papers (2024-06-26T15:09:54Z) - Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking.
We propose MoMA-M3T, a framework that mainly consists of three motion-aware components.
We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
arXiv Detail & Related papers (2023-08-22T17:53:58Z) - Tracking by 3D Model Estimation of Unknown Objects in Videos [122.56499878291916]
We argue that this representation is limited and instead propose to guide and improve 2D tracking with an explicit object representation.
Our representation tackles a complex long-term dense correspondence problem between all 3D points on the object for all video frames.
The proposed optimization minimizes a novel loss function to estimate the best 3D shape, texture, and 6DoF pose.
arXiv Detail & Related papers (2023-04-13T11:32:36Z) - Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
This paper presents a novel approach that views each tracklet as a continuous stream.
At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank.
To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
arXiv Detail & Related papers (2023-03-14T02:58:27Z) - DirectTracker: 3D Multi-Object Tracking Using Direct Image Alignment and
Photometric Bundle Adjustment [41.27664827586102]
Direct methods have shown excellent performance in the applications of visual odometry and SLAM.
We propose a framework that effectively combines direct image alignment for the short-term tracking and sliding-window photometric bundle adjustment for 3D object detection.
arXiv Detail & Related papers (2022-09-29T17:40:22Z) - MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People
Tracking Benchmark [40.363608495563305]
We provide a large-scale densely-labeled multi-camera tracking dataset in five different environments with the help of an auto-annotation system.
The 3D tracking results are projected to each RGB camera view using camera parameters to create 2D tracking results.
This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments.
arXiv Detail & Related papers (2021-11-30T06:29:14Z) - LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera
Multi-Object Tracking [42.87953709286856]
Multi-Camera Multi-Object Tracking is currently drawing attention in the computer vision field due to its superior performance in real-world applications.
We propose a mathematically elegant multi-camera multiple object tracking approach based on a spatial-temporal lifted multicut formulation.
arXiv Detail & Related papers (2021-11-23T14:09:47Z) - VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the
Wild [98.69191256693703]
We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines.
It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment.
It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.
arXiv Detail & Related papers (2021-08-05T08:35:44Z) - Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving.
We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
arXiv Detail & Related papers (2021-03-12T15:30:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.