Monocular Quasi-Dense 3D Object Tracking

• URL: http://arxiv.org/abs/2103.07351v1
• Date: Fri, 12 Mar 2021 15:30:02 GMT
• Title: Monocular Quasi-Dense 3D Object Tracking
• Authors: Hou-Ning Hu, Yung-Hsu Yang, Tobias Fischer, Trevor Darrell, Fisher Yu, Min Sun
• Abstract summary: A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
• Score: 99.51683944057191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform. The object association leverages quasi-dense similarity learning to identify objects in various poses and viewpoints with appearance cues only. After initial 2D association, we further utilize 3D bounding boxes depth-ordering heuristics for robust instance association and motion-based 3D trajectory prediction for re-identification of occluded vehicles. In the end, an LSTM-based object velocity learning module aggregates the long-term trajectory information for more accurate motion extrapolation. Experiments on our proposed simulation data and real-world benchmarks, including KITTI, nuScenes, and Waymo datasets, show that our tracking framework offers robust object association and tracking on urban-driving scenarios. On the Waymo Open benchmark, we establish the first camera-only baseline in the 3D tracking and 3D detection challenges. Our quasi-dense 3D tracking pipeline achieves impressive improvements on the nuScenes 3D tracking benchmark with near five times tracking accuracy of the best vision-only submission among all published methods. Our code, data and trained models are available at https://github.com/SysCV/qd-3dt.

Related papers

• TAPVid-3D: A Benchmark for Tracking Any Point in 3D [63.060421798990845]
  We introduce a new benchmark, TAPVid-3D, for evaluating the task of Tracking Any Point in 3D. This benchmark will serve as a guidepost to improve our ability to understand precise 3D motion and surface deformation from monocular video.
  arXiv  Detail & Related papers  (2024-07-08T13:28:47Z)
• Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
  We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking. We propose MoMA-M3T, a framework that mainly consists of three motion-aware components. We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-22T17:53:58Z)
• Tracking by 3D Model Estimation of Unknown Objects in Videos [122.56499878291916]
  We argue that this representation is limited and instead propose to guide and improve 2D tracking with an explicit object representation. Our representation tackles a complex long-term dense correspondence problem between all 3D points on the object for all video frames. The proposed optimization minimizes a novel loss function to estimate the best 3D shape, texture, and 6DoF pose.
  arXiv  Detail & Related papers  (2023-04-13T11:32:36Z)
• Time3D: End-to-End Joint Monocular 3D Object Detection and Tracking for Autonomous Driving [3.8073142980733]
  We propose jointly training 3D detection and 3D tracking from only monocular videos in an end-to-end manner. Time3D achieves 21.4% AMOTA, 13.6% AMOTP on the nuScenes 3D tracking benchmark, surpassing all published competitors.
  arXiv  Detail & Related papers  (2022-05-30T06:41:10Z)
• A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds [50.54083964183614]
  It is non-trivial to perform accurate target-specific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. We propose DMT, a Detector-free Motion prediction based 3D Tracking network that totally removes the usage of complicated 3D detectors.
  arXiv  Detail & Related papers  (2022-03-08T17:49:07Z)
• 3D Visual Tracking Framework with Deep Learning for Asteroid Exploration [22.808962211830675]
  In this paper, we focus on the studied accurate and real-time method for 3D tracking. A new large-scale 3D asteroid tracking dataset is presented, including binocular video sequences, depth maps, and point clouds of diverse asteroids. We propose a deep-learning based 3D tracking framework, named as Track3D, which involves 2D monocular tracker and a novel light-weight amodal axis-aligned bounding-box network, A3BoxNet.
  arXiv  Detail & Related papers  (2021-11-21T04:14:45Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation [8.854112907350624]
  3D multi-object tracking plays a vital role in autonomous navigation. Many approaches detect objects in 2D RGB sequences for tracking, which is lack of reliability when localizing objects in 3D space. We propose a novel convolutional operation, named RelationConv, to better exploit the correlation between each pair of objects in the adjacent frames.
  arXiv  Detail & Related papers  (2020-11-25T16:14:40Z)
• Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking [34.40019455462043]
  We propose to learn 3D object correspondences from temporal point cloud data and infer the motion information from correspondence patterns. Our method exceeds the existing 3D tracking methods on both the KITTI and larger scale Nuscenes dataset.
  arXiv  Detail & Related papers  (2020-10-20T06:07:20Z)
• Tracking Emerges by Looking Around Static Scenes, with Neural 3D Mapping [23.456046776979903]
  We propose to leverage multiview data of textitstatic points in arbitrary scenes (static or dynamic) to learn a neural 3D mapping module. The neural 3D mapper consumes RGB-D data as input, and produces a 3D voxel grid of deep features as output. We show that our unsupervised 3D object trackers outperform prior unsupervised 2D and 2.5D trackers, and approach the accuracy of supervised trackers.
  arXiv  Detail & Related papers  (2020-08-04T02:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.