A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing

• URL: http://arxiv.org/abs/2609.08338v2
• Date: Thu, 10 Sep 2026 10:32:57 GMT
• Title: A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing
• Abstract summary: This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in an autonomous racing domain.<n>Independent detections from cameras, LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles.<n>The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout.
• Score: 1.3281177137699656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object detection and tracking are fundamental components of perception systems for autonomous driving. Achieving robust performance under adverse conditions such as limited visibility, sensor noise, and failures remains an open challenge, particularly in autonomous racing, where vehicles operate at very high speeds, experience strong vibrations, and interact under small safety margins. This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in the autonomous racing domain. The proposed system extends previous work by exploiting all onboard sensors through a late-fusion approach and a dedicated multi-object tracking framework. Independent detections from cameras, LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles. The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout. Experimental evaluation on real-world data across diverse critical scenarios, representative of challenging edge cases also in urban driving, confirms the effectiveness of the proposed pipeline and its suitability to support safe and adaptive planning decisions.

Related papers

• One-Stage Object Detectors in Autonomous Driving [0.0]
  This paper presents a survey and analysis of one-stage object detectors for autonomous driving.<n>It reviews the evolution of major one-stage detectors, including YOLOv1, SSD, RetinaNet, EfficientDet, anchor-free detectors such as FCOS and CenterNet, and recent real-time models such as YOLOv10.<n>Overall, this survey highlights how one-stage detectors balance speed, accuracy, efficiency, and robustness, while also emphasizing the remaining gap between benchmark results and dependable real-world autonomous-driving performance.
  arXiv  Detail & Related papers  (2026-08-19T15:13:33Z)
• Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations [122.22391796628408]
  We present Bench2Drive-Robust, to our knowledge the first device-centric robustness benchmark for closed-loop end-to-end autonomous driving.<n>We systematically evaluate deployment-oriented perturbations arising from three major sources: camera-stream failures, ego-state estimation errors, and compute-induced control delay.<n>Our results show that these deployment-related perturbations can substantially degrade closed-loop driving performance.
  arXiv  Detail & Related papers  (2026-05-18T08:45:24Z)
• AVOID: The Adverse Visual Conditions Dataset with Obstacles for Driving Scene Understanding [48.97660297411286]
  We introduce AVOID, a new dataset for real-time obstacle detection in a simulated environment.<n>AVOID consists of a large set of unexpected road obstacles located along each path captured under various weather and time conditions.<n>Each image is coupled with the corresponding semantic and depth maps, raw and semantic LiDAR data, and waypoints.
  arXiv  Detail & Related papers  (2025-12-29T05:34:26Z)
• Classification of Driver Behaviour Using External Observation Techniques for Autonomous Vehicles [0.7734726150561086]
  This study introduces a novel driver behaviour classification system that uses external observation techniques to detect indicators of distraction and impairment.<n>The proposed framework employs advanced computer vision methodologies, including real-time object tracking, lateral displacement analysis, and lane position monitoring.<n>Unlike systems reliant on inter-vehicular communication, this vision-based approach enables behavioural analysis of non-connected vehicles.
  arXiv  Detail & Related papers  (2025-09-11T11:05:14Z)
• LiDAR-Based Vehicle Detection and Tracking for Autonomous Racing [0.8356765961526956]
  This paper presents the LiDAR-based perception algorithms deployed on Team PoliMOVE's autonomous racecar, which won multiple competitions in the Indy Autonomous Challenge series.<n> Experimental results demonstrate the algorithm's performance, robustness, computational efficiency, and suitability for autonomous racing applications.
  arXiv  Detail & Related papers  (2025-01-24T14:01:51Z)
• CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus Attentions [13.981748780317329]
  Accurately and promptly predicting accidents among surrounding traffic agents from camera footage is crucial for the safety of autonomous vehicles (AVs) This study introduces a novel accident anticipation framework for AVs, termed CRASH. It seamlessly integrates five components: object detector, feature extractor, object-aware module, context-aware module, and multi-layer fusion. Our model surpasses existing top baselines in critical evaluation metrics like Average Precision (AP) and mean Time-To-Accident (mTTA)
  arXiv  Detail & Related papers  (2024-07-25T04:12:49Z)
• Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors [6.166992288822812]
  Multi-Object Tracking plays a critical role in ensuring safer and more efficient navigation through complex traffic scenarios. This paper presents a novel deep learning-based method that integrates radar and camera data to enhance the accuracy and robustness of Multi-Object Tracking in autonomous driving systems.
  arXiv  Detail & Related papers  (2024-07-10T21:09:09Z)
• OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising [49.86409475232849]
  Trajectory prediction is fundamental in computer vision and autonomous driving. Existing approaches in this field often assume precise and complete observational data. We present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique.
  arXiv  Detail & Related papers  (2024-04-02T18:30:29Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics [77.34726150561087]
  This work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. We give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.
  arXiv  Detail & Related papers  (2023-03-08T00:48:32Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.