A Point-Based Approach to Efficient LiDAR Multi-Task Perception

• URL: http://arxiv.org/abs/2404.12798v1
• Date: Fri, 19 Apr 2024 11:24:34 GMT
• Title: A Point-Based Approach to Efficient LiDAR Multi-Task Perception
• Authors: Christopher Lang, Alexander Braun, Lars Schillingmann, Abhinav Valada,
• Abstract summary: PAttFormer is an efficient multi-task architecture for joint semantic segmentation and object detection in point clouds. Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for task-specific point cloud representations. Our evaluations show substantial gains from multi-task learning, improving LiDAR semantic segmentation by +1.7% in mIou and 3D object detection by +1.7% in mAP.
• Score: 49.91741677556553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-task networks can potentially improve performance and computational efficiency compared to single-task networks, facilitating online deployment. However, current multi-task architectures in point cloud perception combine multiple task-specific point cloud representations, each requiring a separate feature encoder and making the network structures bulky and slow. We propose PAttFormer, an efficient multi-task architecture for joint semantic segmentation and object detection in point clouds that only relies on a point-based representation. The network builds on transformer-based feature encoders using neighborhood attention and grid-pooling and a query-based detection decoder using a novel 3D deformable-attention detection head design. Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for multiple task-specific point cloud representations, resulting in a network that is 3x smaller and 1.4x faster while achieving competitive performance on the nuScenes and KITTI benchmarks for autonomous driving perception. Our extensive evaluations show substantial gains from multi-task learning, improving LiDAR semantic segmentation by +1.7% in mIou and 3D object detection by +1.7% in mAP on the nuScenes benchmark compared to the single-task models.

Related papers

• RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception [64.80760846124858]
  This paper proposes a novel unified representation, RepVF, which harmonizes the representation of various perception tasks. RepVF characterizes the structure of different targets in the scene through a vector field, enabling a single-head, multi-task learning model. Building upon RepVF, we introduce RFTR, a network designed to exploit the inherent connections between different tasks.
  arXiv  Detail & Related papers  (2024-07-15T16:25:07Z)
• Multi-task Learning with 3D-Aware Regularization [55.97507478913053]
  We propose a structured 3D-aware regularizer which interfaces multiple tasks through the projection of features extracted from an image encoder to a shared 3D feature space. We show that the proposed method is architecture agnostic and can be plugged into various prior multi-task backbones to improve their performance.
  arXiv  Detail & Related papers  (2023-10-02T08:49:56Z)
• Efficient Controllable Multi-Task Architectures [85.76598445904374]
  We propose a multi-task model consisting of a shared encoder and task-specific decoders where both encoder and decoder channel widths are slimmable. Our key idea is to control the task importance by varying the capacities of task-specific decoders, while controlling the total computational cost. This improves overall accuracy by allowing a stronger encoder for a given budget, increases control over computational cost, and delivers high-quality slimmed sub-architectures.
  arXiv  Detail & Related papers  (2023-08-22T19:09:56Z)
• LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving [7.137567622606353]
  We present a real-time multi-task convolutional neural network for LiDAR-based object detection, semantics, and motion segmentation. We propose a novel Semantic Weighting and Guidance (SWAG) module to transfer semantic features for improved object detection selectively. We achieve state-of-the-art results for two tasks, semantic and motion segmentation, and close to state-of-the-art performance for 3D object detection.
  arXiv  Detail & Related papers  (2023-07-17T21:22:17Z)
• A Dynamic Feature Interaction Framework for Multi-task Visual Perception [100.98434079696268]
  We devise an efficient unified framework to solve multiple common perception tasks. These tasks include instance segmentation, semantic segmentation, monocular 3D detection, and depth estimation. Our proposed framework, termed D2BNet, demonstrates a unique approach to parameter-efficient predictions for multi-task perception.
  arXiv  Detail & Related papers  (2023-06-08T09:24:46Z)
• LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception [15.785527155108966]
  LidarMultiNet is a LiDAR-based multi-task network that unifies 3D object detection, semantic segmentation, and panoptic segmentation. At the core of LidarMultiNet is a strong 3D voxel-based encoder-decoder architecture with a Global Context Pooling (GCP) module. LidarMultiNet is extensively tested on both Open dataset and nuScenes dataset.
  arXiv  Detail & Related papers  (2022-09-19T23:39:15Z)
• A Unified Object Motion and Affinity Model for Online Multi-Object Tracking [127.5229859255719]
  We propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA. UMA integrates single object tracking and metric learning into a unified triplet network by means of multi-task learning. We equip our model with a task-specific attention module, which is used to boost task-aware feature learning.
  arXiv  Detail & Related papers  (2020-03-25T09:36:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.