3D Single-object Tracking in Point Clouds with High Temporal Variation

• URL: http://arxiv.org/abs/2408.02049v3
• Date: Fri, 6 Sep 2024 07:48:03 GMT
• Title: 3D Single-object Tracking in Point Clouds with High Temporal Variation
• Authors: Qiao Wu, Kun Sun, Pei An, Mathieu Salzmann, Yanning Zhang, Jiaqi Yang,
• Abstract summary: High temporal variation of point clouds is the key challenge of 3D single-object tracking (3D SOT) Existing approaches rely on the assumption that the shape variation of the point clouds and the motion of the objects across neighboring frames are smooth. We present a novel framework for 3D SOT in point clouds with high temporal variation, called HVTrack.
• Score: 79.5863632942935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The high temporal variation of the point clouds is the key challenge of 3D single-object tracking (3D SOT). Existing approaches rely on the assumption that the shape variation of the point clouds and the motion of the objects across neighboring frames are smooth, failing to cope with high temporal variation data. In this paper, we present a novel framework for 3D SOT in point clouds with high temporal variation, called HVTrack. HVTrack proposes three novel components to tackle the challenges in the high temporal variation scenario: 1) A Relative-Pose-Aware Memory module to handle temporal point cloud shape variations; 2) a Base-Expansion Feature Cross-Attention module to deal with similar object distractions in expanded search areas; 3) a Contextual Point Guided Self-Attention module for suppressing heavy background noise. We construct a dataset with high temporal variation (KITTI-HV) by setting different frame intervals for sampling in the KITTI dataset. On the KITTI-HV with 5 frame intervals, our HVTrack surpasses the state-of-the-art tracker CXTracker by 11.3%/15.7% in Success/Precision.

Related papers

• SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking [26.405519771454102]
  We introduce Sequence-to-Sequence tracking paradigm and a tracker named SeqTrack3D to capture target motion across continuous frames. This novel method ensures robust tracking by leveraging location priors from historical boxes, even in scenes with sparse points. Experiments conducted on large-scale datasets show that SeqTrack3D achieves new state-of-the-art performances.
  arXiv  Detail & Related papers  (2024-02-26T02:14:54Z)
• PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection [66.94819989912823]
  We propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection. We use point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement. We conduct extensive experiments on the large-scale dataset to demonstrate that our approach performs well against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-13T18:59:13Z)
• STTracker: Spatio-Temporal Tracker for 3D Single Object Tracking [11.901758708579642]
  3D single object tracking with point clouds is a critical task in 3D computer vision. Previous methods usually input the last two frames and use the template point cloud in previous frame and the search area point cloud in the current frame respectively.
  arXiv  Detail & Related papers  (2023-06-30T07:25:11Z)
• TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object Detection [47.941714033657675]
  3D object detection using point clouds has attracted increasing attention due to its wide applications in autonomous driving and robotics. We design TransPillars, a novel transformer-based feature aggregation technique that exploits temporal features of consecutive point cloud frames. Our proposed TransPillars achieves state-of-art performance as compared to existing multi-frame detection approaches.
  arXiv  Detail & Related papers  (2022-08-04T15:41:43Z)
• Learning Spatial and Temporal Variations for 4D Point Cloud Segmentation [0.39373541926236766]
  We argue that the temporal information across the frames provides crucial knowledge for 3D scene perceptions. We design a temporal variation-aware module and a temporal voxel-point refiner to capture the temporal variation in the 4D point cloud.
  arXiv  Detail & Related papers  (2022-07-11T07:36:26Z)
• IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment [58.8330387551499]
  We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) We propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually.
  arXiv  Detail & Related papers  (2022-03-22T10:14:08Z)
• M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers [78.48081972698888]
  We present M3DeTR, which combines different point cloud representations with different feature scales based on multi-scale feature pyramids. M3DeTR is the first approach that unifies multiple point cloud representations, feature scales, as well as models mutual relationships between point clouds simultaneously using transformers.
  arXiv  Detail & Related papers  (2021-04-24T06:48:23Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.