DriveAdapter: Breaking the Coupling Barrier of Perception and Planning
in End-to-End Autonomous Driving
- URL: http://arxiv.org/abs/2308.00398v2
- Date: Sat, 26 Aug 2023 03:47:35 GMT
- Title: DriveAdapter: Breaking the Coupling Barrier of Perception and Planning
in End-to-End Autonomous Driving
- Authors: Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu,
Hongyang Li
- Abstract summary: State-of-the-art methods usually follow the Teacher-Student' paradigm.
Student model only has access to raw sensor data and conducts behavior cloning on the data collected by the teacher model.
We propose DriveAdapter, which employs adapters with the feature alignment objective function between the student (perception) and teacher (planning) modules.
- Score: 64.57963116462757
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: End-to-end autonomous driving aims to build a fully differentiable system
that takes raw sensor data as inputs and directly outputs the planned
trajectory or control signals of the ego vehicle. State-of-the-art methods
usually follow the `Teacher-Student' paradigm. The Teacher model uses
privileged information (ground-truth states of surrounding agents and map
elements) to learn the driving strategy. The student model only has access to
raw sensor data and conducts behavior cloning on the data collected by the
teacher model. By eliminating the noise of the perception part during planning
learning, state-of-the-art works could achieve better performance with
significantly less data compared to those coupled ones.
However, under the current Teacher-Student paradigm, the student model still
needs to learn a planning head from scratch, which could be challenging due to
the redundant and noisy nature of raw sensor inputs and the casual confusion
issue of behavior cloning. In this work, we aim to explore the possibility of
directly adopting the strong teacher model to conduct planning while letting
the student model focus more on the perception part. We find that even equipped
with a SOTA perception model, directly letting the student model learn the
required inputs of the teacher model leads to poor driving performance, which
comes from the large distribution gap between predicted privileged inputs and
the ground-truth.
To this end, we propose DriveAdapter, which employs adapters with the feature
alignment objective function between the student (perception) and teacher
(planning) modules. Additionally, since the pure learning-based teacher model
itself is imperfect and occasionally breaks safety rules, we propose a method
of action-guided feature learning with a mask for those imperfect teacher
features to further inject the priors of hand-crafted rules into the learning
process.
Related papers
- A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving [21.130543517747995]
This paper introduces the Human-Like Trajectory Prediction (H) model, which adopts a teacher-student knowledge distillation framework.
The "teacher" model mimics the visual processing of the human brain, particularly the functions of the occipital and temporal lobes.
The "student" model focuses on real-time interaction and decision-making, capturing essential perceptual cues for accurate prediction.
arXiv Detail & Related papers (2024-02-29T15:22:26Z) - L2T-DLN: Learning to Teach with Dynamic Loss Network [4.243592852049963]
In existing works, the teacher iteration model 1) merely determines the loss function based on the present states of the student model.
In this paper, we first formulate the loss adjustment as a temporal task by designing a teacher model with memory units.
Then, with a dynamic loss network, we can additionally use the states of the loss to assist the teacher learning in enhancing the interactions between the teacher and the student model.
arXiv Detail & Related papers (2023-10-30T07:21:40Z) - Coaching a Teachable Student [10.81020059614133]
We propose a knowledge distillation framework for teaching a sensorimotor student agent to drive from the supervision of a privileged teacher agent.
Key insight is to design a student which learns to align their input features with the teacher's privileged Bird's Eye View (BEV) space.
To scaffold the difficult sensorimotor learning task, the student model is optimized via a student-paced coaching mechanism with various auxiliary supervision.
arXiv Detail & Related papers (2023-06-16T17:59:38Z) - Distantly-Supervised Named Entity Recognition with Adaptive Teacher
Learning and Fine-grained Student Ensemble [56.705249154629264]
Self-training teacher-student frameworks are proposed to improve the robustness of NER models.
In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks.
Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
arXiv Detail & Related papers (2022-12-13T12:14:09Z) - Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
We propose self-distillation as a regularization for a further pre-training stage.
We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
arXiv Detail & Related papers (2022-09-30T02:25:12Z) - Oracle Teacher: Leveraging Target Information for Better Knowledge
Distillation of CTC Models [10.941519846908697]
We introduce a new type of teacher model for connectionist temporal classification ( CTC)-based sequence models, namely Oracle Teacher.
Since the Oracle Teacher learns a more accurate CTC alignment by referring to the target information, it can provide the student with more optimal guidance.
Based on a many-to-one mapping property of the CTC algorithm, we present a training strategy that can effectively prevent the trivial solution.
arXiv Detail & Related papers (2021-11-05T14:14:05Z) - Decentralized Federated Learning Preserves Model and Data Privacy [77.454688257702]
We propose a fully decentralized approach, which allows to share knowledge between trained models.
Students are trained on the output of their teachers via synthetically generated input data.
The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher.
arXiv Detail & Related papers (2021-02-01T14:38:54Z) - Action-Based Representation Learning for Autonomous Driving [8.296684637620551]
We propose to use action-based driving data for learning representations.
Our experiments show that an affordance-based driving model pre-trained with this approach can leverage a relatively small amount of weakly annotated imagery.
arXiv Detail & Related papers (2020-08-21T10:49:13Z) - Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy.
We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space.
We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
arXiv Detail & Related papers (2020-07-14T16:42:59Z) - Learning to Reweight with Deep Interactions [104.68509759134878]
We propose an improved data reweighting algorithm, in which the student model provides its internal states to the teacher model.
Experiments on image classification with clean/noisy labels and neural machine translation empirically demonstrate that our algorithm makes significant improvement over previous methods.
arXiv Detail & Related papers (2020-07-09T09:06:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.