Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

• URL: http://arxiv.org/abs/2007.13971v1
• Date: Tue, 28 Jul 2020 03:33:41 GMT
• Title: Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions
• Abstract summary: This paper describes the challenges and solutions to build a low-latency, high-accuracy perception system for DUT18 Driverless. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.
• Score: 3.093385446319071
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which it raced. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We highlight modifications required to adapt perception CNNs to racing domains, improvements to loss functions used for pose estimation, and methodologies for sub-microsecond camera synchronization among other improvements. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.

Related papers

• Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations [122.22391796628408]
  We present Bench2Drive-Robust, to our knowledge the first device-centric robustness benchmark for closed-loop end-to-end autonomous driving.<n>We systematically evaluate deployment-oriented perturbations arising from three major sources: camera-stream failures, ego-state estimation errors, and compute-induced control delay.<n>Our results show that these deployment-related perturbations can substantially degrade closed-loop driving performance.
  arXiv  Detail & Related papers  (2026-05-18T08:45:24Z)
• Vision-Based Lane Following and Traffic Sign Recognition for Resource-Constrained Autonomous Vehicles [0.0]
  This paper presents a vision-based framework that integrates lane detection, lane tracking, and traffic sign recognition for embedded autonomous vehicles.<n>A rule-based steering controller generates steering commands to maintain stable vehicle navigation.<n>Experiments show that the system achieves real-time performance while maintaining accurate lane tracking with only 3.16% maximum offset RMSE.
  arXiv  Detail & Related papers  (2026-04-23T20:19:32Z)
• DriveFlow: Rectified Flow Adaptation for Robust 3D Object Detection in Autonomous Driving [85.14946767994932]
  DriveFlow is a Rectified Flow Adaptation method for training data enhancement in autonomous driving.<n>It incorporates a high-frequency alignment loss for foreground to maintain precise 3D object geometry.<n>It also conducts dual-frequency optimization for background, balancing editing flexibility and semantic consistency.
  arXiv  Detail & Related papers  (2025-11-24T03:12:43Z)
• Mini Autonomous Car Driving based on 3D Convolutional Neural Networks [0.6349729476511409]
  Development of reliable and trustworthy autonomous systems poses challenges such as high complexity, prolonged training periods, and intrinsic levels of uncertainty.<n>Mini Autonomous Cars (MACs) are used as a practical testbed, enabling validation of autonomous control methodologies on small-scale setups.<n>This work presents a methodology based on RGB-D information and three-dimensional convolutional neural networks (3D CNNs) for MAC autonomous driving in simulated environments.
  arXiv  Detail & Related papers  (2025-08-29T00:21:34Z)
• Vision-based Perception for Autonomous Vehicles in Obstacle Avoidance Scenarios [0.0]
  We propose an efficient obstacle avoidance pipeline that leverages a camera-only perception module and a Frenet-Pure Pursuit-based planning strategy.<n>By integrating advancements in computer vision, the system utilizes YOLOv11 for object detection and state-of-the-art monocular depth estimation models, such as Depth Anything V2, to estimate object distances.<n>The system is evaluated in diverse scenarios on a university campus, demonstrating its effectiveness in handling various obstacles and enhancing autonomous navigation.
  arXiv  Detail & Related papers  (2025-07-16T17:41:14Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up. It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention. It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• LiDAR-Based Vehicle Detection and Tracking for Autonomous Racing [0.8356765961526956]
  This paper presents the LiDAR-based perception algorithms deployed on Team PoliMOVE's autonomous racecar, which won multiple competitions in the Indy Autonomous Challenge series. Experimental results demonstrate the algorithm's performance, robustness, computational efficiency, and suitability for autonomous racing applications.
  arXiv  Detail & Related papers  (2025-01-24T14:01:51Z)
• Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning [13.613407983544427]
  We introduce a robust model designed to withstand changes in camera position within the vehicle. Our Driver Behavior Monitoring Network (DBMNet) relies on a lightweight backbone and integrates a disentanglement module. Experiments conducted on the daytime and nighttime subsets of the 100-Driver dataset validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-11-20T10:27:12Z)
• Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition [8.841708075914353]
  Temporal localization of driving actions over time is important for advanced driver-assistance systems and naturalistic driving studies. We aim to improve the temporal localization and classification accuracy performance by adapting video action recognition and 2D human-based estimation networks to one model. The model performs well on the A2 test set the 2023 NVIDIA AI City Challenge for naturalistic driving action recognition.
  arXiv  Detail & Related papers  (2024-03-11T10:26:38Z)
• Beyond the Mud: Datasets and Benchmarks for Computer Vision in Off-Road Racing [38.719032057630024]
  Off-road motorcycle Racer Number dataset and Muddy Racer re-iDentification dataset feature over 6,300 images taken during off-road competitions. Off-the-shelf models transfer poorly, reaching only 15% end-to-end (E2E) F1 score on text spotting, and 33% rank-1 accuracy on ReID. Fine-tuning yields major improvements, bringing model performance to 53% F1 score for E2E text spotting and 79% rank-1 accuracy on ReID.
  arXiv  Detail & Related papers  (2024-02-12T19:42:05Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Anticipating Driving Behavior through Deep Learning-Based Policy Prediction [66.344923925939]
  We developed a system that processes integrated visual features derived from video frames captured by a regular camera, along with depth details obtained from a point cloud scanner. This system is designed to anticipate driving actions, encompassing both vehicle speed and steering angle. Our evaluations indicate that the forecasts achieve a noteworthy level of accuracy in a minimum of half the test scenarios.
  arXiv  Detail & Related papers  (2023-07-20T17:38:55Z)
• FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving [18.02943016671203]
  This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes. In particular, we introduce a Full-Scale depth prediction network named FSNet. With FSNet, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data.
  arXiv  Detail & Related papers  (2023-04-21T03:17:04Z)
• Motion Planning and Control for Multi Vehicle Autonomous Racing at High Speeds [100.61456258283245]
  This paper presents a multi-layer motion planning and control architecture for autonomous racing. The proposed solution has been applied on a Dallara AV-21 racecar and tested at oval race tracks achieving lateral accelerations up to 25 $m/s2$.
  arXiv  Detail & Related papers  (2022-07-22T15:16:54Z)
• Indy Autonomous Challenge -- Autonomous Race Cars at the Handling Limits [81.22616193933021]
  The team TUM Auton-omous Motorsports will participate in the Indy Autonomous Challenge in Octo-ber 2021. It will benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapolis Motor Speedway. It is an ideal testing ground for the development of autonomous driving algorithms capable of mastering the most challenging and rare situations.
  arXiv  Detail & Related papers  (2022-02-08T11:55:05Z)
• Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing [70.16043883381677]
  In autonomous racing, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions. We introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors.
  arXiv  Detail & Related papers  (2022-01-10T10:02:40Z)
• Real-time Streaming Perception System for Autonomous Driving [2.6058660721533187]
  We present the real-time steaming perception system, which is also the 2nd Place solution of Streaming Perception Challenge. Unlike traditional object detection challenges, which focus mainly on the absolute performance, streaming perception task requires achieving a balance of accuracy and latency. On the Argoverse-HD test set, our method achieves 33.2 streaming AP (34.6 streaming AP verified by the organizer) under the required hardware.
  arXiv  Detail & Related papers  (2021-07-30T01:32:44Z)
• Towards Autonomous Driving: a Multi-Modal 360$^{\circ}$ Perception Proposal [87.11988786121447]
  This paper presents a framework for 3D object detection and tracking for autonomous vehicles. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack.
  arXiv  Detail & Related papers  (2020-08-21T20:36:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.