SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2508.10567v1
• Date: Thu, 14 Aug 2025 12:02:41 GMT
• Title: SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving
• Authors: Philipp Wolters, Johannes Gilg, Torben Teepe, Gerhard Rigoll,
• Abstract summary: SpaRC-AD is a query-based end-to-end camera-radar fusion framework for planning-oriented autonomous driving.<n>Our method achieves strong improvements over the state-of-the-art vision-only baselines across multiple autonomous driving tasks.
• Score: 5.343552118560704
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end autonomous driving systems promise stronger performance through unified optimization of perception, motion forecasting, and planning. However, vision-based approaches face fundamental limitations in adverse weather conditions, partial occlusions, and precise velocity estimation - critical challenges in safety-sensitive scenarios where accurate motion understanding and long-horizon trajectory prediction are essential for collision avoidance. To address these limitations, we propose SpaRC-AD, a query-based end-to-end camera-radar fusion framework for planning-oriented autonomous driving. Through sparse 3D feature alignment, and doppler-based velocity estimation, we achieve strong 3D scene representations for refinement of agent anchors, map polylines and motion modelling. Our method achieves strong improvements over the state-of-the-art vision-only baselines across multiple autonomous driving tasks, including 3D detection (+4.8% mAP), multi-object tracking (+8.3% AMOTA), online mapping (+1.8% mAP), motion prediction (-4.0% mADE), and trajectory planning (-0.1m L2 and -9% TPC). We achieve both spatial coherence and temporal consistency on multiple challenging benchmarks, including real-world open-loop nuScenes, long-horizon T-nuScenes, and closed-loop simulator Bench2Drive. We show the effectiveness of radar-based fusion in safety-critical scenarios where accurate motion understanding and long-horizon trajectory prediction are essential for collision avoidance. The source code of all experiments is available at https://phi-wol.github.io/sparcad/

Related papers

• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• Tracking Meets Large Multimodal Models for Driving Scenario Understanding [76.71815464110153]
  Large Multimodal Models (LMMs) have recently gained prominence in autonomous driving research.<n>We propose to integrate tracking information as an additional input to recover 3D spatial and temporal details.<n>We introduce a novel approach for embedding this tracking information into LMMs to enhance their understanding of driving scenarios.
  arXiv  Detail & Related papers  (2025-03-18T17:59:12Z)
• Bayesian Approximation-Based Trajectory Prediction and Tracking with 4D Radar [13.438311878715536]
  3D multi-object tracking (MOT) is vital for autonomous vehicles, yet LiDAR and camera-based methods degrade in adverse weather.<n>We propose Bayes-4DRTrack, a 4D Radar-based MOT framework that adopts a transformer-based motion prediction network.
  arXiv  Detail & Related papers  (2025-02-03T13:49:21Z)
• MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategies [12.485905108032146]
  This paper introduces MetaOcc, a novel multi-modal framework for omni-oriented 3D occupancy prediction.<n>To address the limitations of directly applying encoders to sparse radar data, we propose a Radar Height Self-Attention module.<n>To reduce reliance on expensive point cloud, we propose a pseudo-label generation pipeline based on an open-set segmentor.
  arXiv  Detail & Related papers  (2025-01-26T03:51:56Z)
• Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors [6.166992288822812]
  Multi-Object Tracking plays a critical role in ensuring safer and more efficient navigation through complex traffic scenarios. This paper presents a novel deep learning-based method that integrates radar and camera data to enhance the accuracy and robustness of Multi-Object Tracking in autonomous driving systems.
  arXiv  Detail & Related papers  (2024-07-10T21:09:09Z)
• Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark [23.872360763782037]
  ASAP is the first benchmark to evaluate the online performance of vision-centric perception in autonomous driving. We propose an annotation-extending pipeline to generate high-frame-rate labels for the 12Hz raw images. In the ASAP benchmark, comprehensive experiment results reveal that the model rank alters under different constraints.
  arXiv  Detail & Related papers  (2022-12-17T16:32:15Z)
• Motion Planning and Control for Multi Vehicle Autonomous Racing at High Speeds [100.61456258283245]
  This paper presents a multi-layer motion planning and control architecture for autonomous racing. The proposed solution has been applied on a Dallara AV-21 racecar and tested at oval race tracks achieving lateral accelerations up to 25 $m/s2$.
  arXiv  Detail & Related papers  (2022-07-22T15:16:54Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• IntentNet: Learning to Predict Intention from Raw Sensor Data [86.74403297781039]
  In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
  arXiv  Detail & Related papers  (2021-01-20T00:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.