REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View

• URL: http://arxiv.org/abs/2605.11824v1
• Date: Tue, 12 May 2026 09:13:02 GMT
• Title: REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View
• Abstract summary: We address the challenge of multimodal sensor fusion by aligning radar and camera data in a unified domain.<n>We employ a variational encoder-decoder architecture that learns the transformation of front-view camera data into the Bird's-Eye View (BEV) polar domain.<n>This alignment ensures that both modalities are represented in a compatible domain, robust and efficient sensor fusion.
• Score: 13.463025176847703
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: A realistic view of the vehicle's surroundings is generally offered by camera sensors, which is crucial for environmental perception. Affordable radar sensors, on the other hand, are becoming invaluable due to their robustness in variable weather conditions. However, because of their noisy output and reduced classification capability, they work best when combined with other sensor data. Specifically, we address the challenge of multimodal sensor fusion by aligning radar and camera data in a unified domain, prioritizing not only accuracy, but also computational efficiency. Our work leverages the raw range-Doppler (RD) spectrum from radar and front-view camera images as inputs. To enable effective fusion, we employ a variational encoder-decoder architecture that learns the transformation of front-view camera data into the Bird's-Eye View (BEV) polar domain. Concurrently, a radar encoder-decoder learns to recover the angle information from the RD data that produce Range-Azimuth (RA) features. This alignment ensures that both modalities are represented in a compatible domain, facilitating robust and efficient sensor fusion. We evaluated our fusion strategy for vehicle detection and free space segmentation against state-of-the-art methods using the RADIal dataset.

Related papers

• RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields [49.41438378775056]
  Radars are an ideal complement to cameras, with cameras offering fine angular resolution and radars providing metric depth and robustness under adverse weather.<n>RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.
  arXiv  Detail & Related papers  (2026-05-25T21:03:40Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data [7.2508100569856975]
  We use the raw range-Doppler spectrum of radar data to process camera images. We extract the corresponding features with our camera encoder-decoder architecture. The resultant feature maps are fused with Range-Azimuth features, recovered from the RD spectrum input to perform object detection.
  arXiv  Detail & Related papers  (2024-11-20T13:26:13Z)
• DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection [0.7919810878571297]
  We propose a novel camera-radar fusion approach called Dual Perspective Fusion Transformer (DPFT)<n>Our method leverages lower-level radar data (the radar cube) instead of the processed point clouds to preserve as much information as possible.<n>DPFT has demonstrated state-of-the-art performance on the K-Radar dataset while showing remarkable robustness against adverse weather conditions.
  arXiv  Detail & Related papers  (2024-04-03T18:54:27Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework [10.931114142452895]
  Current SOTA algorithm combines Camera and Lidar sensors, limited by the high price of Lidar. HVDetFusion is a multi-modal detection algorithm that supports pure camera data as input for detection, but also can perform fusion input of radar data and camera data. HVDetFusion achieves the new state-of-the-art 67.4% NDS on the challenging nuScenes test set among all camera-radar 3D object detectors.
  arXiv  Detail & Related papers  (2023-07-21T03:08:28Z)
• CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection [12.557361522985898]
  We propose a camera-radar matching network CramNet to fuse the sensor readings from camera and radar in a joint 3D space. Our method supports training with sensor modality dropout, which leads to robust 3D object detection, even when a camera or radar sensor suddenly malfunctions on a vehicle.
  arXiv  Detail & Related papers  (2022-10-17T17:18:47Z)
• Complex-valued Convolutional Neural Networks for Enhanced Radar Signal Denoising and Interference Mitigation [73.0103413636673]
  We propose the use of Complex-Valued Convolutional Neural Networks (CVCNNs) to address the issue of mutual interference between radar sensors. CVCNNs increase data efficiency, speeds up network training and substantially improves the conservation of phase information during interference removal.
  arXiv  Detail & Related papers  (2021-04-29T10:06:29Z)
• All-Weather Object Recognition Using Radar and Infrared Sensing [1.7513645771137178]
  This thesis explores new sensing developments based on long wave polarised infrared (IR) imagery and imaging radar to recognise objects. First, we developed a methodology based on Stokes parameters using polarised infrared data to recognise vehicles using deep neural networks. Second, we explored the potential of using only the power spectrum captured by low-THz radar sensors to perform object recognition in a controlled scenario. Last, we created a new large-scale dataset in the "wild" with many different weather scenarios showing radar robustness to detect vehicles in adverse weather.
  arXiv  Detail & Related papers  (2020-10-30T14:16:39Z)
• Depth Estimation from Monocular Images and Sparse Radar Data [93.70524512061318]
  In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions.
  arXiv  Detail & Related papers  (2020-09-30T19:01:33Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.