MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection

• URL: http://arxiv.org/abs/2507.04369v1
• Date: Sun, 06 Jul 2025 12:29:45 GMT
• Title: MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection
• Authors: Hanshi Wang, Jin Gao, Weiming Hu, Zhipeng Zhang,
• Abstract summary: We present the first work demonstrating that a pure Mamba block can achieve efficient Dense Global Fusion.<n>Our motivation stems from the observation that existing fusion strategies are constrained by their inability to simultaneously achieve efficiency.<n>We propose height-fidelity LiDAR encoding that preserves precise height information through voxel compression in continuous space.
• Score: 45.792346999032496
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the first work demonstrating that a pure Mamba block can achieve efficient Dense Global Fusion, meanwhile guaranteeing top performance for camera-LiDAR multi-modal 3D object detection. Our motivation stems from the observation that existing fusion strategies are constrained by their inability to simultaneously achieve efficiency, long-range modeling, and retaining complete scene information. Inspired by recent advances in state-space models (SSMs) and linear attention, we leverage their linear complexity and long-range modeling capabilities to address these challenges. However, this is non-trivial since our experiments reveal that simply adopting efficient linear-complexity methods does not necessarily yield improvements and may even degrade performance. We attribute this degradation to the loss of height information during multi-modal alignment, leading to deviations in sequence order. To resolve this, we propose height-fidelity LiDAR encoding that preserves precise height information through voxel compression in continuous space, thereby enhancing camera-LiDAR alignment. Subsequently, we introduce the Hybrid Mamba Block, which leverages the enriched height-informed features to conduct local and global contextual learning. By integrating these components, our method achieves state-of-the-art performance with the top-tire NDS score of 75.0 on the nuScenes validation benchmark, even surpassing methods that utilize high-resolution inputs. Meanwhile, our method maintains efficiency, achieving faster inference speed than most recent state-of-the-art methods.

Related papers

• MambaNeXt-YOLO: A Hybrid State Space Model for Real-time Object Detection [4.757840725810513]
  YOLO-series models have set strong benchmarks by balancing speed and accuracy.<n>Transformers have high computational complexity because of their self-attention mechanism.<n>We propose MambaNeXt-YOLO, a novel object detection framework that balances accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-06-04T07:46:24Z)
• S3MOT: Monocular 3D Object Tracking with Selective State Space Model [3.5047603107971397]
  Multi-object tracking in 3D space is essential for advancing robotics and computer applications.<n>It remains a significant challenge in monocular setups due to the difficulty of mining 3D associations from 2D video streams.<n>We present three innovative techniques to enhance the fusion of heterogeneous cues for monocular 3D MOT.
  arXiv  Detail & Related papers  (2025-04-25T04:45:35Z)
• Efficient Multimodal 3D Object Detector via Instance-Level Contrastive Distillation [17.634678949648208]
  We introduce a fast yet effective multimodal 3D object detector, incorporating our proposed Instance-level Contrastive Distillation (ICD) framework and Cross Linear Attention Fusion Module (CLFM)<n>Our 3D object detector outperforms state-of-the-art (SOTA) methods while achieving superior efficiency.
  arXiv  Detail & Related papers  (2025-03-17T08:26:11Z)
• MobileMamba: Lightweight Multi-Receptive Visual Mamba Network [51.33486891724516]
  Previous research on lightweight models has primarily focused on CNNs and Transformer-based designs. We propose the MobileMamba framework, which balances efficiency and performance. MobileMamba achieves up to 83.6% on Top-1, surpassing existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-24T18:01:05Z)
• Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation [22.653014803666668]
  We propose a Faster LiDAR 3D object detection framework, called FASD, which implements heterogeneous model distillation by adaptively uniform cross-model voxel features. We aim to distill the transformer's capacity for high-performance sequence modeling into Mamba models with low FLOPs, achieving a significant improvement in accuracy through knowledge transfer. We evaluated the framework on datasets and nuScenes, achieving a 4x reduction in resource consumption and a 1-2% performance improvement over the current SoTA methods.
  arXiv  Detail & Related papers  (2024-09-17T09:30:43Z)
• Mamba-based Light Field Super-Resolution with Efficient Subspace Scanning [48.99361249764921]
  Transformer-based methods have demonstrated impressive performance in 4D light field (LF) super-resolution. However, their quadratic complexity hinders the efficient processing of high resolution 4D inputs. We propose a Mamba-based Light Field Super-Resolution method, named MLFSR, by designing an efficient subspace scanning strategy.
  arXiv  Detail & Related papers  (2024-06-23T11:28:08Z)
• MLF-DET: Multi-Level Fusion for Cross-Modal 3D Object Detection [54.52102265418295]
  We propose a novel and effective Multi-Level Fusion network, named as MLF-DET, for high-performance cross-modal 3D object DETection. For the feature-level fusion, we present the Multi-scale Voxel Image fusion (MVI) module, which densely aligns multi-scale voxel features with image features. For the decision-level fusion, we propose the lightweight Feature-cued Confidence Rectification (FCR) module, which exploits image semantics to rectify the confidence of detection candidates.
  arXiv  Detail & Related papers  (2023-07-18T11:26:02Z)
• GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds [72.60362979456035]
  Masked Autoencoders (MAE) are challenging to explore in large-scale 3D point clouds. We propose a textbfGenerative textbfDecoder for MAE (GD-MAE) to automatically merges the surrounding context. We demonstrate the efficacy of the proposed method on several large-scale benchmarks: KITTI, and ONCE.
  arXiv  Detail & Related papers  (2022-12-06T14:32:55Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Exploring Data Augmentation for Multi-Modality 3D Object Detection [82.9988604088494]
  It is counter-intuitive that multi-modality methods based on point cloud and images perform only marginally better or sometimes worse than approaches that solely use point cloud. We propose a pipeline, named transformation flow, to bridge the gap between single and multi-modality data augmentation with transformation reversing and replaying. Our method also wins the best PKL award in the 3rd nuScenes detection challenge.
  arXiv  Detail & Related papers  (2020-12-23T15:23:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.