SMamba: Sparse Mamba for Event-based Object Detection

• URL: http://arxiv.org/abs/2501.11971v1
• Date: Tue, 21 Jan 2025 08:33:32 GMT
• Title: SMamba: Sparse Mamba for Event-based Object Detection
• Authors: Nan Yang, Yang Wang, Zhanwen Liu, Meng Li, Yisheng An, Xiangmo Zhao,
• Abstract summary: Transformer-based methods have achieved remarkable performance in event-based object detection, owing to the global modeling ability.<n>To mitigate cost, some researchers propose window attention based sparsification strategies to discard unimportant regions.<n>We propose Sparse Mamba, which performs adaptive sparsification to reduce computational effort while maintaining global modeling ability.
• Score: 17.141967728323714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformer-based methods have achieved remarkable performance in event-based object detection, owing to the global modeling ability. However, they neglect the influence of non-event and noisy regions and process them uniformly, leading to high computational overhead. To mitigate computation cost, some researchers propose window attention based sparsification strategies to discard unimportant regions, which sacrifices the global modeling ability and results in suboptimal performance. To achieve better trade-off between accuracy and efficiency, we propose Sparse Mamba (SMamba), which performs adaptive sparsification to reduce computational effort while maintaining global modeling capability. Specifically, a Spatio-Temporal Continuity Assessment module is proposed to measure the information content of tokens and discard uninformative ones by leveraging the spatiotemporal distribution differences between activity and noise events. Based on the assessment results, an Information-Prioritized Local Scan strategy is designed to shorten the scan distance between high-information tokens, facilitating interactions among them in the spatial dimension. Furthermore, to extend the global interaction from 2D space to 3D representations, a Global Channel Interaction module is proposed to aggregate channel information from a global spatial perspective. Results on three datasets (Gen1, 1Mpx, and eTram) demonstrate that our model outperforms other methods in both performance and efficiency.

Related papers

• DP-LET: An Efficient Spatio-Temporal Network Traffic Prediction Framework [13.65226228907662]
  DP-LET is an efficient feature-temporal network traffic prediction framework. DP-LET consists of a data processing module, a local feature enhancement module, and a Transformer-based prediction module. A real-world cellular traffic prediction demonstrates the practicality of DP-LET.
  arXiv  Detail & Related papers  (2025-04-04T02:52:43Z)
• Can foundation models actively gather information in interactive environments to test hypotheses? [56.651636971591536]
  We introduce a framework in which a model must determine the factors influencing a hidden reward function.<n>We investigate whether approaches such as self- throughput and increased inference time improve information gathering efficiency.
  arXiv  Detail & Related papers  (2024-12-09T12:27:21Z)
• Multilateral Cascading Network for Semantic Segmentation of Large-Scale Outdoor Point Clouds [6.253217784798542]
  Multilateral Cascading Network (MCNet) designed to address this challenge.<n>MCNet comprises two key components: a Multilateral Cascading Attention Enhancement (MCAE) module, and a Point Cross Stage Partial (P-CSP) module.<n>Our results surpassed the current best result by 2.1% in overall mIoU and yielded an improvement of 15.9% on average for small-sample object categories.
  arXiv  Detail & Related papers  (2024-09-21T02:23:01Z)
• AMMUNet: Multi-Scale Attention Map Merging for Remote Sensing Image Segmentation [4.618389486337933]
  We propose AMMUNet, a UNet-based framework that employs multi-scale attention map merging. The proposed AMMM effectively combines multi-scale attention maps into a unified representation using a fixed mask template. We show that our approach achieves remarkable mean intersection over union (mIoU) scores of 75.48% on the Vaihingen dataset and an exceptional 77.90% on the Potsdam dataset.
  arXiv  Detail & Related papers  (2024-04-20T15:23:15Z)
• DistFormer: Enhancing Local and Global Features for Monocular Per-Object Distance Estimation [35.6022448037063]
  Per-object distance estimation is crucial in safety-critical applications such as autonomous driving, surveillance, and robotics. Existing approaches rely on two scales: local information (i.e., the bounding box proportions) or global information. Our work aims to strengthen both local and global cues.
  arXiv  Detail & Related papers  (2024-01-06T10:56:36Z)
• X Modality Assisting RGBT Object Tracking [1.730147049648545]
  A novel X Modality Assisting Network (X-Net) is introduced, which explores the impact of the fusion paradigm by decoupling visual object tracking into three distinct levels. X-Net achieves performance gains of 0.47%/1.2% in the average of precise rate and success rate.
  arXiv  Detail & Related papers  (2023-12-27T05:38:54Z)
• Generalizing Event-Based Motion Deblurring in Real-World Scenarios [62.995994797897424]
  Event-based motion deblurring has shown promising results by exploiting low-latency events. We propose a scale-aware network that allows flexible input spatial scales and enables learning from different temporal scales of motion blur. A two-stage self-supervised learning scheme is then developed to fit real-world data distribution.
  arXiv  Detail & Related papers  (2023-08-11T04:27:29Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.