PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection

• URL: http://arxiv.org/abs/2312.08371v2
• Date: Wed, 24 Apr 2024 17:53:08 GMT
• Title: PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection
• Authors: Kuan-Chih Huang, Weijie Lyu, Ming-Hsuan Yang, Yi-Hsuan Tsai,
• Abstract summary: We propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection. We use point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement. We conduct extensive experiments on the large-scale dataset to demonstrate that our approach performs well against state-of-the-art methods.
• Score: 66.94819989912823
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent temporal LiDAR-based 3D object detectors achieve promising performance based on the two-stage proposal-based approach. They generate 3D box candidates from the first-stage dense detector, followed by different temporal aggregation methods. However, these approaches require per-frame objects or whole point clouds, posing challenges related to memory bank utilization. Moreover, point clouds and trajectory features are combined solely based on concatenation, which may neglect effective interactions between them. In this paper, we propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection. To this end, we only utilize point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement. Furthermore, we introduce modules to encode trajectory features, focusing on long short-term and future-aware perspectives, and then effectively aggregate them with point cloud features. We conduct extensive experiments on the large-scale Waymo dataset to demonstrate that our approach performs well against state-of-the-art methods. Code and models will be made publicly available at https://github.com/kuanchihhuang/PTT.

Related papers

• LEF: Late-to-Early Temporal Fusion for LiDAR 3D Object Detection [40.267769862404684]
  We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings into the early stages of a 3D object detector.
  arXiv  Detail & Related papers  (2023-09-28T21:58:25Z)
• TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses [51.60422927416087]
  3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots. We present TrajectoryFormer, a novel point-cloud-based 3D MOT framework.
  arXiv  Detail & Related papers  (2023-06-09T13:31:50Z)
• MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences [38.7464958249103]
  We propose MoDAR, using motion forecasting outputs as a type of virtual modality, to augment LiDAR point clouds. A fused point cloud of both raw sensor points and the virtual points can then be fed to any off-the-shelf point-cloud based 3D object detector.
  arXiv  Detail & Related papers  (2023-06-05T19:28:19Z)
• Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
  This paper presents a novel approach that views each tracklet as a continuous stream. At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank. To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
  arXiv  Detail & Related papers  (2023-03-14T02:58:27Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object Detection [47.941714033657675]
  3D object detection using point clouds has attracted increasing attention due to its wide applications in autonomous driving and robotics. We design TransPillars, a novel transformer-based feature aggregation technique that exploits temporal features of consecutive point cloud frames. Our proposed TransPillars achieves state-of-art performance as compared to existing multi-frame detection approaches.
  arXiv  Detail & Related papers  (2022-08-04T15:41:43Z)
• LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention [100.52873557168637]
  3D object detectors usually focus on the single-frame detection, while ignoring the information in consecutive point cloud frames. In this paper, we propose an end-to-end online 3D video object detector that operates on point sequences.
  arXiv  Detail & Related papers  (2020-04-03T06:06:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.