End-to-End Driving via Self-Supervised Imitation Learning Using Camera and LiDAR Data

• URL: http://arxiv.org/abs/2308.14329v2
• Date: Thu, 31 Oct 2024 07:16:46 GMT
• Title: End-to-End Driving via Self-Supervised Imitation Learning Using Camera and LiDAR Data
• Authors: Jin Bok Park, Jinkyu Lee, Muhyun Back, Hyunmin Han, David T. Ma, Sang Min Won, Sung Soo Hwang, Il Yong Chun,
• Abstract summary: This letter proposes the first fully self-supervised learning framework, self-supervised imitation learning (SSIL), for E2E driving. The proposed SSIL framework can learn E2E driving networks without using driving command data. Our numerical experiments with three different benchmark datasets demonstrate that the proposed SSIL framework achieves very comparable E2E driving accuracy with the supervised learning counterpart.
• Score: 6.849144123909844
• License:
• Abstract: In autonomous driving, the end-to-end (E2E) driving approach that predicts vehicle control signals directly from sensor data is rapidly gaining attention. To learn a safe E2E driving system, one needs an extensive amount of driving data and human intervention. Vehicle control data is constructed by many hours of human driving, and it is challenging to construct large vehicle control datasets. Often, publicly available driving datasets are collected with limited driving scenes, and collecting vehicle control data is only available by vehicle manufacturers. To address these challenges, this letter proposes the first fully self-supervised learning framework, self-supervised imitation learning (SSIL), for E2E driving, based on the self-supervised regression learning framework. The proposed SSIL framework can learn E2E driving networks without using driving command data. To construct pseudo steering angle data, proposed SSIL predicts a pseudo target from the vehicle's poses at the current and previous time points that are estimated with light detection and ranging sensors. In addition, we propose two modified E2E driving networks that predict driving commands depending on high-level instruction. Our numerical experiments with three different benchmark datasets demonstrate that the proposed SSIL framework achieves very comparable E2E driving accuracy with the supervised learning counterpart.

Related papers

• DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving [81.04174379726251]
  This paper collects a comprehensive end-to-end driving dataset named DriveCoT. It contains sensor data, control decisions, and chain-of-thought labels to indicate the reasoning process. We propose a baseline model called DriveCoT-Agent, trained on our dataset, to generate chain-of-thought predictions and final decisions.
  arXiv  Detail & Related papers  (2024-03-25T17:59:01Z)
• V2X-Lead: LiDAR-based End-to-End Autonomous Driving with Vehicle-to-Everything Communication Integration [4.166623313248682]
  This paper presents a LiDAR-based end-to-end autonomous driving method with Vehicle-to-Everything (V2X) communication integration. The proposed method aims to handle imperfect partial observations by fusing the onboard LiDAR sensor and V2X communication data.
  arXiv  Detail & Related papers  (2023-09-26T20:26:03Z)
• DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle [7.642646077340124]
  DeepIPCv2 is an autonomous driving model that perceives the environment using a LiDAR sensor for more robust drivability. DeepIPCv2 takes a set of LiDAR point clouds as the main perception input.
  arXiv  Detail & Related papers  (2023-07-13T09:23:21Z)
• Unsupervised Driving Event Discovery Based on Vehicle CAN-data [62.997667081978825]
  This work presents a simultaneous clustering and segmentation approach for vehicle CAN-data that identifies common driving events in an unsupervised manner. We evaluate our approach with a dataset of real Tesla Model 3 vehicle CAN-data and a two-hour driving session that we annotated with different driving events.
  arXiv  Detail & Related papers  (2023-01-12T13:10:47Z)
• A Distributed Acoustic Sensor System for Intelligent Transportation using Deep Learning [2.1219631216034127]
  This work explores a novel data source based on optical fibre-based distributed acoustic sensors (DAS) for traffic analysis. We propose a deep learning technique to analyse DAS signals to address this challenge through continuous sensing and without exposing personal information. We achieve 92% vehicle classification accuracy and 92%-97% in occupancy detection based on DAS data collected under controlled conditions.
  arXiv  Detail & Related papers  (2022-09-13T13:23:30Z)
• Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
  In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle. In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
  arXiv  Detail & Related papers  (2022-07-05T16:33:20Z)
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [54.61668577827041]
  We introduce COOPERNAUT, an end-to-end learning model that uses cross-vehicle perception for vision-based cooperative driving. Our experiments on AutoCastSim suggest that our cooperative perception driving models lead to a 40% improvement in average success rate.
  arXiv  Detail & Related papers  (2022-05-04T17:55:12Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)
• IntentNet: Learning to Predict Intention from Raw Sensor Data [86.74403297781039]
  In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
  arXiv  Detail & Related papers  (2021-01-20T00:31:52Z)
• Vehicle Route Prediction through Multiple Sensors Data Fusion [0.0]
  The framework consists of two modules and both are working in sequence. The first module of our framework using a deep learning for recognizing the vehicle license plate number. The second module using supervised learning algorithm of machine learning for predicting the route of the vehicle.
  arXiv  Detail & Related papers  (2020-08-30T08:14:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.