MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D
Temporal Object Detection
- URL: http://arxiv.org/abs/2205.05979v1
- Date: Thu, 12 May 2022 09:38:42 GMT
- Title: MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D
Temporal Object Detection
- Authors: Xuesong Chen, Shaoshuai Shi, Benjin Zhu, Ka Chun Cheung, Hang Xu and
Hongsheng Li
- Abstract summary: We present a flexible and high-performance 3D detection framework, named MPPNet, for 3D temporal object detection with point cloud sequences.
We propose a novel three-hierarchy framework with proxy points for multi-frame feature encoding and interactions to achieve better detection.
Our approach outperforms state-of-the-art methods with large margins when applied to both short (e.g., 4-frame) and long (e.g., 16-frame) point cloud sequences.
- Score: 44.619039588252676
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Accurate and reliable 3D detection is vital for many applications including
autonomous driving vehicles and service robots. In this paper, we present a
flexible and high-performance 3D detection framework, named MPPNet, for 3D
temporal object detection with point cloud sequences. We propose a novel
three-hierarchy framework with proxy points for multi-frame feature encoding
and interactions to achieve better detection. The three hierarchies conduct
per-frame feature encoding, short-clip feature fusion, and whole-sequence
feature aggregation, respectively. To enable processing long-sequence point
clouds with reasonable computational resources, intra-group feature mixing and
inter-group feature attention are proposed to form the second and third feature
encoding hierarchies, which are recurrently applied for aggregating multi-frame
trajectory features. The proxy points not only act as consistent object
representations for each frame, but also serve as the courier to facilitate
feature interaction between frames. The experiments on largeWaymo Open dataset
show that our approach outperforms state-of-the-art methods with large margins
when applied to both short (e.g., 4-frame) and long (e.g., 16-frame) point
cloud sequences. Specifically, MPPNet achieves 74.21%, 74.62% and 73.31% for
vehicle, pedestrian and cyclist classes on the LEVEL 2 mAPH metric with
16-frame input.
Related papers
- PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection [66.94819989912823]
We propose a point-trajectory transformer with long short-term memory for efficient temporal 3D object detection.
We use point clouds of current-frame objects and their historical trajectories as input to minimize the memory bank storage requirement.
We conduct extensive experiments on the large-scale dataset to demonstrate that our approach performs well against state-of-the-art methods.
arXiv Detail & Related papers (2023-12-13T18:59:13Z) - MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection
from Point Cloud Sequences [21.50329070835023]
Point cloud sequences are commonly used to accurately detect 3D objects in applications such as autonomous driving.
Current top-performing multi-frame detectors mostly follow a Detect-and-Fuse framework, which extracts features from each frame of the sequence and fuses them to detect the objects in the current frame.
We propose an efficient Motion-guided Sequential Fusion (MSF) method, which exploits the continuity of object motion to mine useful sequential contexts for object detection in the current frame.
arXiv Detail & Related papers (2023-03-15T02:10:27Z) - Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
This paper presents a novel approach that views each tracklet as a continuous stream.
At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank.
To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
arXiv Detail & Related papers (2023-03-14T02:58:27Z) - 3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D
Point Clouds [95.54285993019843]
We propose a method for joint detection and tracking of multiple objects in 3D point clouds.
Our model exploits temporal information employing multiple frames to detect objects and track them in a single network.
arXiv Detail & Related papers (2022-11-01T20:59:38Z) - TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object
Detection [47.941714033657675]
3D object detection using point clouds has attracted increasing attention due to its wide applications in autonomous driving and robotics.
We design TransPillars, a novel transformer-based feature aggregation technique that exploits temporal features of consecutive point cloud frames.
Our proposed TransPillars achieves state-of-art performance as compared to existing multi-frame detection approaches.
arXiv Detail & Related papers (2022-08-04T15:41:43Z) - Background-Aware 3D Point Cloud Segmentationwith Dynamic Point Feature
Aggregation [12.093182949686781]
We propose a novel 3D point cloud learning network, referred to as Dynamic Point Feature Aggregation Network (DPFA-Net)
DPFA-Net has two variants for semantic segmentation and classification of 3D point clouds.
It achieves the state-of-the-art overall accuracy score for semantic segmentation on the S3DIS dataset.
arXiv Detail & Related papers (2021-11-14T05:46:05Z) - Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose
Estimation [61.98690211671168]
We propose a Multi-level Attention-Decoder Network (MAED) to model multi-level attentions in a unified framework.
With the training set of 3DPW, MAED outperforms previous state-of-the-art methods by 6.2, 7.2, and 2.4 mm of PA-MPJPE.
arXiv Detail & Related papers (2021-09-06T09:06:17Z) - Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking
from View Aggregation [8.854112907350624]
3D multi-object tracking plays a vital role in autonomous navigation.
Many approaches detect objects in 2D RGB sequences for tracking, which is lack of reliability when localizing objects in 3D space.
We propose a novel convolutional operation, named RelationConv, to better exploit the correlation between each pair of objects in the adjacent frames.
arXiv Detail & Related papers (2020-11-25T16:14:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.