Mask-RadarNet: Enhancing Transformer With Spatial-Temporal Semantic Context for Radar Object Detection in Autonomous Driving

• URL: http://arxiv.org/abs/2412.15595v1
• Date: Fri, 20 Dec 2024 06:39:40 GMT
• Title: Mask-RadarNet: Enhancing Transformer With Spatial-Temporal Semantic Context for Radar Object Detection in Autonomous Driving
• Abstract summary: We propose a model called Mask-RadarNet to fully utilize the hierarchical semantic features from the input radar data.<n>Mask-RadarNet exploits the combination of interleaved convolution and attention operations to replace the traditional architecture in transformer-based models.<n>With relatively lower computational complexity and fewer parameters, the proposed Mask-RadarNet achieves higher recognition accuracy for object detection in autonomous driving.
• Score: 11.221694136475554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a cost-effective and robust technology, automotive radar has seen steady improvement during the last years, making it an appealing complement to commonly used sensors like camera and LiDAR in autonomous driving. Radio frequency data with rich semantic information are attracting more and more attention. Most current radar-based models take radio frequency image sequences as the input. However, these models heavily rely on convolutional neural networks and leave out the spatial-temporal semantic context during the encoding stage. To solve these problems, we propose a model called Mask-RadarNet to fully utilize the hierarchical semantic features from the input radar data. Mask-RadarNet exploits the combination of interleaved convolution and attention operations to replace the traditional architecture in transformer-based models. In addition, patch shift is introduced to the Mask-RadarNet for efficient spatial-temporal feature learning. By shifting part of patches with a specific mosaic pattern in the temporal dimension, Mask-RadarNet achieves competitive performance while reducing the computational burden of the spatial-temporal modeling. In order to capture the spatial-temporal semantic contextual information, we design the class masking attention module (CMAM) in our encoder. Moreover, a lightweight auxiliary decoder is added to our model to aggregate prior maps generated from the CMAM. Experiments on the CRUW dataset demonstrate the superiority of the proposed method to some state-of-the-art radar-based object detection algorithms. With relatively lower computational complexity and fewer parameters, the proposed Mask-RadarNet achieves higher recognition accuracy for object detection in autonomous driving.

Related papers

• IRGNN: Efficient Invariant Radar Graph Neural Network for Radar Point Cloud Object Detection [6.250476866823555]
  We propose IRGNN, an Invariant Radar Graph Neural Network for radar point cloud object detection.<n>IRGNN first reconstructs radar point clouds into graph representations using translation- and rotation-invariant feature designs.<n>It then employs an improved message passing neural network (MPNN) with residual connections and a virtual node layer to enhance local feature propagation and global context modeling.
  arXiv  Detail & Related papers  (2026-08-14T15:35:06Z)
• RAVEN: Radar Adaptive Vision Encoders for Efficient Chirp-wise Object Detection and Segmentation [11.345174599707859]
  RAVEN is a computationally efficient deep learning architecture for FMCW radar perception.<n>Across automotive radar benchmarks, the approach reports strong object detection and BEV free-space segmentation performance.
  arXiv  Detail & Related papers  (2026-04-06T07:30:45Z)
• Leveraging Transformer Decoder for Automotive Radar Object Detection [9.764772760421792]
  We present a Transformer-based architecture for 3D radar object detection that uses a novel Transformer Decoder.<n> Pyramid Token Fusion (PTF) converts a feature pyramid into a unified, scale-aware token sequence.<n>We evaluate the proposed framework on the RADDet, where it achieves significant improvements over state-of-the-art radar-only baselines.
  arXiv  Detail & Related papers  (2026-01-19T20:44:24Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• mRadNet: A Compact Radar Object Detector with MetaFormer [0.4666493857924357]
  We propose mRadNet, a novel radar object detection model with compactness in mind.<n>mRadNet employs a U-net style architecture with MetaFormer blocks, in which separable convolution and attention token mixers are used.<n>The performance of mRadNet is validated on the CRUW dataset, improving state-of-the-art performance with the least number of parameters and FLOPs.
  arXiv  Detail & Related papers  (2025-09-11T16:34:12Z)
• SparseRadNet: Sparse Perception Neural Network on Subsampled Radar Data [5.344444942640663]
  Radar raw data often contains excessive noise, whereas radar point clouds retain only limited information. We introduce an adaptive subsampling method together with a tailored network architecture that exploits the sparsity patterns. Experiments on the RADIal dataset show that our SparseRadNet exceeds state-of-the-art (SOTA) performance in object detection and achieves close to SOTA accuracy in freespace segmentation.
  arXiv  Detail & Related papers  (2024-06-15T11:26:10Z)
• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• ARFA: An Asymmetric Receptive Field Autoencoder Model for Spatiotemporal Prediction [55.30913411696375]
  We propose an Asymmetric Receptive Field Autoencoder (ARFA) model, which introduces corresponding sizes of receptive field modules. In the encoder, we present large kernel module for globaltemporal feature extraction. In the decoder, we develop a small kernel module for localtemporal reconstruction. We construct the RainBench, a large-scale radar echo dataset for precipitation prediction, to address the scarcity of meteorological data in the domain.
  arXiv  Detail & Related papers  (2023-09-01T07:55:53Z)
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds [59.45414406974091]
  We introduce a deep-learning based method to convolve radar detections into point clouds. We adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds.
  arXiv  Detail & Related papers  (2023-05-22T07:09:35Z)
• Deep Radar Inverse Sensor Models for Dynamic Occupancy Grid Maps [0.0]
  We propose a deep learning-based Inverse Sensor Model (ISM) to learn the mapping from sparse radar detections to polar measurement grids. Our approach is the first one to learn a single-frame measurement grid in the polar scheme from radars with a limited Field Of View. This enables us to flexibly use one or more radar sensors without network retraining and without requirements on 360deg sensor coverage.
  arXiv  Detail & Related papers  (2023-05-21T09:09:23Z)
• A recurrent CNN for online object detection on raw radar frames [7.074916574419171]
  This work presents a new recurrent CNN architecture for online radar object detection. We propose an end-to-end trainable architecture mixing convolutions and ConvLSTMs to learn dependencies between successive frames. Our model is causal and requires only the past information encoded in the memory of the ConvLSTMs to detect objects.
  arXiv  Detail & Related papers  (2022-12-21T16:36:36Z)
• Complex-valued Convolutional Neural Networks for Enhanced Radar Signal Denoising and Interference Mitigation [73.0103413636673]
  We propose the use of Complex-Valued Convolutional Neural Networks (CVCNNs) to address the issue of mutual interference between radar sensors. CVCNNs increase data efficiency, speeds up network training and substantially improves the conservation of phase information during interference removal.
  arXiv  Detail & Related papers  (2021-04-29T10:06:29Z)
• DS-Net: Dynamic Spatiotemporal Network for Video Salient Object Detection [78.04869214450963]
  We propose a novel dynamic temporal-temporal network (DSNet) for more effective fusion of temporal and spatial information. We show that the proposed method achieves superior performance than state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2020-12-09T06:42:30Z)
• Quantized Neural Networks for Radar Interference Mitigation [14.540226579203207]
  CNN-based approaches for denoising and interference mitigation yield promising results for radar processing. We investigate quantization techniques for CNN-based denoising and interference mitigation of radar signals.
  arXiv  Detail & Related papers  (2020-11-25T13:18:06Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.