MambaBEV: An efficient 3D detection model with Mamba2

• URL: http://arxiv.org/abs/2410.12673v1
• Date: Wed, 16 Oct 2024 15:37:29 GMT
• Title: MambaBEV: An efficient 3D detection model with Mamba2
• Authors: Zihan You, Hao Wang, Qichao Zhao, Jinxiang Wang,
• Abstract summary: We propose a mamba2-based BEV 3D object detection model named MambaBEV. We also adapt an end to end self driving paradigm to test the performance of the model.
• Score: 4.782473183865045
• License:
• Abstract: A stable 3D object detection model based on BEV paradigm with temporal information is very important for autonomous driving systems. However, current temporal fusion model use convolutional layer or deformable self-attention is not conducive to the exchange of global information of BEV space and has more computational cost. Recently, a newly proposed based model specialized in processing sequence called mamba has shown great potential in multiple downstream task. In this work, we proposed a mamba2-based BEV 3D object detection model named MambaBEV. We also adapt an end to end self driving paradigm to test the performance of the model. Our work performs pretty good results on nucences datasets:Our base version achieves 51.7% NDS. Our code will be available soon.

Related papers

• BEVWorld: A Multimodal World Model for Autonomous Driving via Unified BEV Latent Space [57.68134574076005]
  We present BEVWorld, a novel approach that tokenizes multimodal sensor inputs into a unified and compact Bird's Eye View latent space for environment modeling. Experiments demonstrate the effectiveness of BEVWorld in autonomous driving tasks, showcasing its capability in generating future scenes and benefiting downstream tasks such as perception and motion prediction.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection [57.019527599167255]
  Multi-view 3D detection based on BEV (bird-eye-view) has recently achieved significant improvements. We show in our paper that directly applying quantization in BEV tasks will 1) make the training unstable, and 2) lead to intolerable performance degradation. Our method QD-BEV enables a novel view-guided distillation (VGD) objective, which can stabilize the quantization-aware training (QAT) while enhancing the model performance.
  arXiv  Detail & Related papers  (2023-08-21T07:06:49Z)
• Knowledge Distillation from 3D to Bird's-Eye-View for LiDAR Semantic Segmentation [6.326177388323946]
  We develop an effective 3D-to-BEV knowledge distillation method that transfers rich knowledge from 3D voxel-based models to BEV-based models. Our framework mainly consists of two modules: the voxel-to-pillar distillation module and the label-weight distillation module. Label-weight distillation helps the model pay more attention to regions with more height information.
  arXiv  Detail & Related papers  (2023-04-22T13:03:19Z)
• MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation [104.12419434114365]
  In real-world applications, sensor corruptions and failures lead to inferior performances. We propose a robust framework, called MetaBEV, to address extreme real-world environments. We show MetaBEV outperforms prior arts by a large margin on both full and corrupted modalities.
  arXiv  Detail & Related papers  (2023-04-19T16:37:17Z)
• DiffBEV: Conditional Diffusion Model for Bird's Eye View Perception [14.968177102647783]
  We propose an end-to-end framework, named DiffBEV, to exploit the potential of diffusion model to generate a more comprehensive BEV representation. In practice, we design three types of conditions to guide the training of the diffusion model which denoises the coarse samples and refines the semantic feature. We show that DiffBEV achieves a 25.9% mIoU on the nuScenes dataset, which is 6.2% higher than the best-performing existing approach.
  arXiv  Detail & Related papers  (2023-03-15T02:42:48Z)
• BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios [51.285561119993105]
  We present BEV-MAE, an efficient masked autoencoder pre-training framework for LiDAR-based 3D object detection in autonomous driving. Specifically, we propose a bird's eye view (BEV) guided masking strategy to guide the 3D encoder learning feature representation. We introduce a learnable point token to maintain a consistent receptive field size of the 3D encoder.
  arXiv  Detail & Related papers  (2022-12-12T08:15:03Z)
• Probabilistic Modeling for Human Mesh Recovery [73.11532990173441]
  This paper focuses on the problem of 3D human reconstruction from 2D evidence. We recast the problem as learning a mapping from the input to a distribution of plausible 3D poses.
  arXiv  Detail & Related papers  (2021-08-26T17:55:11Z)
• SA-Det3D: Self-Attention Based Context-Aware 3D Object Detection [9.924083358178239]
  We propose two variants of self-attention for contextual modeling in 3D object detection. We first incorporate the pairwise self-attention mechanism into the current state-of-the-art BEV, voxel and point-based detectors. Next, we propose a self-attention variant that samples a subset of the most representative features by learning deformations over randomly sampled locations.
  arXiv  Detail & Related papers  (2021-01-07T18:30:32Z)
• PerMO: Perceiving More at Once from a Single Image for Autonomous Driving [76.35684439949094]
  We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image. Our approach combines the strengths of deep learning and the elegance of traditional techniques. We have integrated these algorithms with an autonomous driving system.
  arXiv  Detail & Related papers  (2020-07-16T05:02:45Z)
• An LSTM-Based Autonomous Driving Model Using Waymo Open Dataset [7.151393153761375]
  This paper introduces an approach to learn a short-term memory (LSTM)-based model for imitating the behavior of a self-driving model. The experimental results show that our model outperforms several models in driving action prediction.
  arXiv  Detail & Related papers  (2020-02-14T05:28:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.