Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection

• URL: http://arxiv.org/abs/2602.20632v1
• Date: Tue, 24 Feb 2026 07:25:53 GMT
• Title: Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection
• Abstract summary: SIFormer is a scene-instance aware transformer for 3D object detection using 4D radar and camera.<n>Experiments demonstrate that SIFormer achieves state-of-the-art performance on View-of-Delft, TJ4DRadSet and NuScenes datasets.
• Score: 19.99225197463389
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 4D millimeter-wave radar has emerged as a promising sensing modality for autonomous driving due to its robustness and affordability. However, its sparse and weak geometric cues make reliable instance activation difficult, limiting the effectiveness of existing radar-camera fusion paradigms. BEV-level fusion offers global scene understanding but suffers from weak instance focus, while perspective-level fusion captures instance details but lacks holistic context. To address these limitations, we propose SIFormer, a scene-instance aware transformer for 3D object detection using 4D radar and camera. SIFormer first suppresses background noise during view transformation through segmentation- and depth-guided localization. It then introduces a cross-view activation mechanism that injects 2D instance cues into BEV space, enabling reliable instance awareness under weak radar geometry. Finally, a transformer-based fusion module aggregates complementary image semantics and radar geometry for robust perception. As a result, with the aim of enhancing instance awareness, SIFormer bridges the gap between the two paradigms, combining their complementary strengths to address inherent sparse nature of radar and improve detection accuracy. Experiments demonstrate that SIFormer achieves state-of-the-art performance on View-of-Delft, TJ4DRadSet and NuScenes datasets. Source code is available at github.com/shawnnnkb/SIFormer.

Related papers

• 4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception [12.807651354967183]
  Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout.<n>We propose a 4D radar-camera framework for 360$circ$ full-scene perception, which models semantic occupancy as a persistent scene state.<n>We extend ManTruckScenes with satellite-map-based generated occupancy labels and pair it with OmniHD-Scenes in a unified cross-dataset detection-and-occupancy protocol.
  arXiv  Detail & Related papers  (2026-07-10T17:26:19Z)
• RCGDet3D: Rethinking 4D Radar-Camera Fusion-based 3D Object Detection with Enhanced Radar Feature Encoding [3.65399248155853]
  We show that simply enhancing radar feature extraction can achieve comparable or even higher performance than elaborate fusion modules.<n>We propose RCGDet3D, which centers on radar feature encoding and simplifies multi-modal fusion.<n> Experiments on View-of-Delft (VoD) and TJ4DRadSet show that RCGDet3D outperforms state-of-the-art methods in both accuracy and speed.
  arXiv  Detail & Related papers  (2026-05-20T12:45:01Z)
• R4Det: 4D Radar-Camera Fusion for High-Performance 3D Object Detection [12.09303986935892]
  3D object detection methods that fuse 4D Radar and camera data confront several challenges.<n>We propose R4Det, which enhances depth estimation quality via the Panoramic Depth Fusion module.<n>Experiments show that R4Det achieves state-of-the-art 3D object detection results on the TJ4DRadSet and VoD datasets.
  arXiv  Detail & Related papers  (2026-03-12T05:41:05Z)
• TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder [66.22997415145467]
  This paper presents a joint completion and detection framework that improves the detection feature in sparse areas.<n> Specifically, we propose TransBridge, a novel transformer-based up-sampling block that fuses the features from the detection and completion networks.<n>The results show that our framework consistently improves end-to-end 3D object detection, with the mean average precision (mAP) ranging from 0.7 to 1.5 across multiple methods.
  arXiv  Detail & Related papers  (2025-12-12T00:08:03Z)
• SFGFusion: Surface Fitting Guided 3D Object Detection with 4D Radar and Camera Fusion [12.877894178462297]
  We introduce SFGFusion, a novel camera-4D imaging radar detection network guided by surface fitting.<n>The explicit surface fitting model enhances spatial representation and cross-modal interaction, enabling more reliable prediction of fine-grained dense depth.<n> Experimental results show that SFGFusion effectively fuses camera and 4D radar features, achieving superior performance on the TJ4DRadSet and view-of-delft (VoD) object detection benchmarks.
  arXiv  Detail & Related papers  (2025-10-22T03:56:27Z)
• MLF-4DRCNet: Multi-Level Fusion with 4D Radar and Camera for 3D Object Detection in Autonomous Driving [31.26862558777292]
  MLF-4DRCNet is a novel framework for 3D object detection via multi-level fusion of 4D radar and camera images.<n>It incorporates the point-, scene-, and proposal-level multi-modal information, enabling comprehensive feature representation.<n>It attains performance comparable to LiDAR-based models on the View-of-Delft dataset.
  arXiv  Detail & Related papers  (2025-09-23T04:02:28Z)
• CVFusion: Cross-View Fusion of 4D Radar and Camera for 3D Object Detection [11.109888378081187]
  We propose a cross-view two-stage fusion network called CVFusion.<n>In the first stage, we design a radar guided iterative (RGIter) BEV fusion module to generate high-recall 3D proposal boxes.<n>In the second stage, we aggregate features from multiple heterogeneous views including points, image, and BEV for each proposal.<n>Our method outperforms the previous state-of-the-art methods by a large margin, with 9.10% and 3.68% mAP improvements on View-of-Delft (VoD) and TJ4DRadSet, respectively.
  arXiv  Detail & Related papers  (2025-07-07T00:45:53Z)
• ZFusion: An Effective Fuser of Camera and 4D Radar for 3D Object Perception in Autonomous Driving [7.037019489455008]
  We propose a 3D object detection method, termed ZFusion, which fuses 4D radar and vision modality.<n> FP-DDCA fuser packs Transformer blocks to interactively fuse multi-modal features at different scales.<n>Experiments show that ZFusion achieved the state-of-the-art mAP in the region of interest.
  arXiv  Detail & Related papers  (2025-04-04T13:29:32Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion [58.77329237533034]
  We propose a Radar-Camera fusion transformer (RaCFormer) to boost the accuracy of 3D object detection.<n>RaCFormer achieves superior results of 64.9% mAP and 70.2% on nuScenes datasets.
  arXiv  Detail & Related papers  (2024-12-17T09:47:48Z)
• Multi-Modal 3D Object Detection by Box Matching [109.43430123791684]
  We propose a novel Fusion network by Box Matching (FBMNet) for multi-modal 3D detection. With the learned assignments between 3D and 2D object proposals, the fusion for detection can be effectively performed by combing their ROI features.
  arXiv  Detail & Related papers  (2023-05-12T18:08:51Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion [45.171150395915056]
  3D semantic scene completion (SSC) is an ill-posed perception task that requires inferring a dense 3D scene from limited observations. Previous camera-based methods struggle to predict accurate semantic scenes due to inherent geometric ambiguity and incomplete observations. We resort to stereo matching technique and bird's-eye-view (BEV) representation learning to address such issues in SSC.
  arXiv  Detail & Related papers  (2023-03-24T12:33:44Z)
• Bridging the View Disparity of Radar and Camera Features for Multi-modal Fusion 3D Object Detection [6.959556180268547]
  This paper focuses on how to utilize millimeter-wave (MMW) radar and camera sensor fusion for 3D object detection. A novel method which realizes the feature-level fusion under bird-eye view (BEV) for a better feature representation is proposed.
  arXiv  Detail & Related papers  (2022-08-25T13:21:37Z)
• Progressive Coordinate Transforms for Monocular 3D Object Detection [52.00071336733109]
  We propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations. In this paper, we propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations.
  arXiv  Detail & Related papers  (2021-08-12T15:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.