Learning to Track with Object Permanence
- URL: http://arxiv.org/abs/2103.14258v1
- Date: Fri, 26 Mar 2021 04:43:04 GMT
- Title: Learning to Track with Object Permanence
- Authors: Pavel Tokmakov, Jie Li, Wolfram Burgard, Adrien Gaidon
- Abstract summary: We introduce an end-to-end trainable approach for joint object detection and tracking.
Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
- Score: 61.36492084090744
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Tracking by detection, the dominant approach for online multi-object
tracking, alternates between localization and re-identification steps. As a
result, it strongly depends on the quality of instantaneous observations, often
failing when objects are not fully visible. In contrast, tracking in humans is
underlined by the notion of object permanence: once an object is recognized, we
are aware of its physical existence and can approximately localize it even
under full occlusions. In this work, we introduce an end-to-end trainable
approach for joint object detection and tracking that is capable of such
reasoning. We build on top of the recent CenterTrack architecture, which takes
pairs of frames as input, and extend it to videos of arbitrary length. To this
end, we augment the model with a spatio-temporal, recurrent memory module,
allowing it to reason about object locations and identities in the current
frame using all the previous history. It is, however, not obvious how to train
such an approach. We study this question on a new, large-scale, synthetic
dataset for multi-object tracking, which provides ground truth annotations for
invisible objects, and propose several approaches for supervising tracking
behind occlusions. Our model, trained jointly on synthetic and real data,
outperforms the state of the art on KITTI, and MOT17 datasets thanks to its
robustness to occlusions.
Related papers
- Leveraging Object Priors for Point Tracking [25.030407197192]
Point tracking is a fundamental problem in computer vision with numerous applications in AR and robotics.
We propose a novel objectness regularization approach that guides points to be aware of object priors.
Our approach achieves state-of-the-art performance on three point tracking benchmarks.
arXiv Detail & Related papers (2024-09-09T16:48:42Z) - Lost and Found: Overcoming Detector Failures in Online Multi-Object Tracking [15.533652456081374]
Multi-object tracking (MOT) endeavors to precisely estimate identities and positions of multiple objects over time.
Modern detectors may occasionally miss some objects in certain frames, causing trackers to cease tracking prematurely.
We propose BUSCA, meaning to search', a versatile framework compatible with any online TbD system.
arXiv Detail & Related papers (2024-07-14T10:45:12Z) - Object-Centric Multiple Object Tracking [124.30650395969126]
This paper proposes a video object-centric model for multiple-object tracking pipelines.
It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module.
Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
arXiv Detail & Related papers (2023-09-01T03:34:12Z) - Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR
based 3D Object Detection [50.959453059206446]
This paper aims for high-performance offline LiDAR-based 3D object detection.
We first observe that experienced human annotators annotate objects from a track-centric perspective.
We propose a high-performance offline detector in a track-centric perspective instead of the conventional object-centric perspective.
arXiv Detail & Related papers (2023-04-24T17:59:05Z) - Standing Between Past and Future: Spatio-Temporal Modeling for
Multi-Camera 3D Multi-Object Tracking [30.357116118917368]
We propose an end-to-end multi-camera 3D multi-object tracking framework.
We name it "Past-and-Future reasoning for Tracking" (PFTrack)
arXiv Detail & Related papers (2023-02-07T23:46:34Z) - SpOT: Spatiotemporal Modeling for 3D Object Tracking [68.12017780034044]
3D multi-object tracking aims to consistently identify all mobile time.
Current 3D tracking methods rely on abstracted information and limited history.
We develop a holistic representation of scenes that leverage both spatial and temporal information.
arXiv Detail & Related papers (2022-07-12T21:45:49Z) - Learning to Track Object Position through Occlusion [32.458623495840904]
Occlusion is one of the most significant challenges encountered by object detectors and trackers.
We propose a tracking-by-detection approach that builds upon the success of region based video object detectors.
Our approach achieves superior results on a dataset of furniture assembly videos collected from the internet.
arXiv Detail & Related papers (2021-06-20T22:29:46Z) - Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using
Spatial and Temporal Transformers [20.806348407522083]
MO3TR is an end-to-end online multi-object tracking framework.
It encodes object interactions into long-term temporal embeddings.
It tracks initiation and termination without the need for an explicit data association module.
arXiv Detail & Related papers (2021-03-27T07:23:38Z) - Detecting Invisible People [58.49425715635312]
We re-purpose tracking benchmarks and propose new metrics for the task of detecting invisible objects.
We demonstrate that current detection and tracking systems perform dramatically worse on this task.
Second, we build dynamic models that explicitly reason in 3D, making use of observations produced by state-of-the-art monocular depth estimation networks.
arXiv Detail & Related papers (2020-12-15T16:54:45Z) - SoDA: Multi-Object Tracking with Soft Data Association [75.39833486073597]
Multi-object tracking (MOT) is a prerequisite for a safe deployment of self-driving cars.
We propose a novel approach to MOT that uses attention to compute track embeddings that encode dependencies between observed objects.
arXiv Detail & Related papers (2020-08-18T03:40:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.