Level 2 Autonomous Driving on a Single Device: Diving into the Devils of Openpilot

• URL: http://arxiv.org/abs/2206.08176v1
• Date: Thu, 16 Jun 2022 13:43:52 GMT
• Title: Level 2 Autonomous Driving on a Single Device: Diving into the Devils of Openpilot
• Authors: Li Chen, Tutian Tang, Zhitian Cai, Yang Li, Penghao Wu, Hongyang Li, Jianping Shi, Junchi Yan, Yu Qiao
• Abstract summary: Comma.ai claims one $999 aftermarket device mounted with a single camera and board inside owns the ability to handle L2 scenarios. Together with open-sourced software of the entire system released by Comma.ai, the project is named Openpilot. In this report, we would like to share our latest findings, shed some light on the new perspective of end-to-end autonomous driving from an industrial product-level side.
• Score: 112.21008828205409
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Equipped with a wide span of sensors, predominant autonomous driving solutions are becoming more modular-oriented for safe system design. Though these sensors have laid a solid foundation, most massive-production solutions up to date still fall into L2 phase. Among these, Comma.ai comes to our sight, claiming one $999 aftermarket device mounted with a single camera and board inside owns the ability to handle L2 scenarios. Together with open-sourced software of the entire system released by Comma.ai, the project is named Openpilot. Is it possible? If so, how is it made possible? With curiosity in mind, we deep-dive into Openpilot and conclude that its key to success is the end-to-end system design instead of a conventional modular framework. The model is briefed as Supercombo, and it can predict the ego vehicle's future trajectory and other road semantics on the fly from monocular input. Unfortunately, the training process and massive amount of data to make all these work are not publicly available. To achieve an intensive investigation, we try to reimplement the training details and test the pipeline on public benchmarks. The refactored network proposed in this work is referred to as OP-Deepdive. For a fair comparison of our version to the original Supercombo, we introduce a dual-model deployment scheme to test the driving performance in the real world. Experimental results on nuScenes, Comma2k19, CARLA, and in-house realistic scenarios verify that a low-cost device can indeed achieve most L2 functionalities and be on par with the original Supercombo model. In this report, we would like to share our latest findings, shed some light on the new perspective of end-to-end autonomous driving from an industrial product-level side, and potentially inspire the community to continue improving the performance. Our code, benchmarks are at https://github.com/OpenPerceptionX/Openpilot-Deepdive.

Related papers

• Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving [59.705635382104454]
  We present Bench2Drive, the first benchmark for evaluating E2E-AD systems' multiple abilities in a closed-loop manner. We implement state-of-the-art E2E-AD models and evaluate them in Bench2Drive, providing insights regarding current status and future directions.
  arXiv  Detail & Related papers  (2024-06-06T09:12:30Z)
• Personalized Autonomous Driving with Large Language Models: Field Experiments [11.429053835807697]
  We introduce an LLM-based framework, Talk2Drive, capable of translating natural verbal commands into executable controls. This is the first-of-its-kind multi-scenario field experiment that deploys LLMs on a real-world autonomous vehicle. We validate that the proposed memory module considers personalized preferences and further reduces the takeover rate by up to 65.2%.
  arXiv  Detail & Related papers  (2023-12-14T23:23:37Z)
• DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving [69.82743399946371]
  DriveMLM is a framework that can perform close-loop autonomous driving in realistic simulators. We employ a multi-modal LLM (MLLM) to model the behavior planning module of a module AD system. This model can plug-and-play in existing AD systems such as Apollo for close-loop driving.
  arXiv  Detail & Related papers  (2023-12-14T18:59:05Z)
• ADriver-I: A General World Model for Autonomous Driving [23.22507419707926]
  We introduce the concept of interleaved vision-action pair, which unifies the format of visual features and control signals. Based on the vision-action pairs, we construct a general world model based on MLLM and diffusion model for autonomous driving, termed ADriver-I. It takes the vision-action pairs as inputs and autoregressively predicts the control signal of the current frame.
  arXiv  Detail & Related papers  (2023-11-22T17:44:29Z)
• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [114.69732301904419]
  We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
  arXiv  Detail & Related papers  (2023-10-26T17:56:35Z)
• Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline [96.31941517446859]
  Current end-to-end autonomous driving methods either run a controller based on a planned trajectory or perform control prediction directly. Our integrated approach has two branches for trajectory planning and direct control, respectively. Results are evaluated in the closed-loop urban driving setting with challenging scenarios using the CARLA simulator.
  arXiv  Detail & Related papers  (2022-06-16T12:42:44Z)
• An Intelligent Self-driving Truck System For Highway Transportation [81.12838700312308]
  In this paper, we introduce an intelligent self-driving truck system. Our presented system consists of three main components, 1) a realistic traffic simulation module for generating realistic traffic flow in testing scenarios, 2) a high-fidelity truck model which is designed and evaluated for mimicking real truck response in real-world deployment. We also deploy our proposed system on a real truck and conduct real world experiments which shows our system's capacity of mitigating sim-to-real gap.
  arXiv  Detail & Related papers  (2021-12-31T04:54:13Z)
• A LiDAR Assisted Control Module with High Precision in Parking Scenarios for Autonomous Driving Vehicle [39.42619778086731]
  We introduce a real-world, industrial scenario of which human drivers are not capable. A precise (3? = 2 centimeters) Error Feedback System was first built to partly replace the localization module. We show that the results not only outperformed original Apollo modules but also beat specially trained and highly experienced human test drivers.
  arXiv  Detail & Related papers  (2021-05-02T06:13:32Z)
• The NVIDIA PilotNet Experiments [5.013775931547319]
  Four years ago, an experimental system known as PilotNet became the first NVIDIA system to steer an autonomous car along a roadway. A single deep neural network (DNN) takes pixels as input and produces a desired vehicle trajectory as output. This document describes the PilotNet lane-keeping effort, carried out over the past five years by our NVIDIA PilotNet group in Holmdel, New Jersey.
  arXiv  Detail & Related papers  (2020-10-17T12:25:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.