Approximating a deep reinforcement learning docking agent using linear model trees

• URL: http://arxiv.org/abs/2203.00369v1
• Date: Tue, 1 Mar 2022 11:32:07 GMT
• Title: Approximating a deep reinforcement learning docking agent using linear model trees
• Authors: Vilde B. Gj{\ae}rum, Ella-Lovise H. R{\o}rvik, Anastasios M. Lekkas
• Abstract summary: linear model tree (LMT) approximates a DNN policy for an autonomous surface vehicle with five control inputs performing a docking operation. LMTs are transparent which makes it possible to associate directly the outputs (control actions) with specific values of the input features. In our simulations, the opaque DNN policy controls the vehicle and the LMT runs in parallel to provide explanations in the form of feature attributions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep reinforcement learning has led to numerous notable results in robotics. However, deep neural networks (DNNs) are unintuitive, which makes it difficult to understand their predictions and strongly limits their potential for real-world applications due to economic, safety, and assurance reasons. To remedy this problem, a number of explainable AI methods have been presented, such as SHAP and LIME, but these can be either be too costly to be used in real-time robotic applications or provide only local explanations. In this paper, the main contribution is the use of a linear model tree (LMT) to approximate a DNN policy, originally trained via proximal policy optimization(PPO), for an autonomous surface vehicle with five control inputs performing a docking operation. The two main benefits of the proposed approach are: a) LMTs are transparent which makes it possible to associate directly the outputs (control actions, in our case) with specific values of the input features, b) LMTs are computationally efficient and can provide information in real-time. In our simulations, the opaque DNN policy controls the vehicle and the LMT runs in parallel to provide explanations in the form of feature attributions. Our results indicate that LMTs can be a useful component within digital assurance frameworks for autonomous ships.

Related papers

• OWLed: Outlier-weighed Layerwise Pruning for Efficient Autonomous Driving Framework [3.8320050452121692]
  We introduce OWLed, the Outlier-Weighed Layerwise Pruning for Efficient Autonomous Driving Framework. Our method assigns non-uniform sparsity ratios to different layers based on the distribution of outlier features. To ensure the compressed model adapts well to autonomous driving tasks, we incorporate driving environment data into both the calibration and pruning processes.
  arXiv  Detail & Related papers  (2024-11-12T10:55:30Z)
• On the Road to Clarity: Exploring Explainable AI for World Models in a Driver Assistance System [3.13366804259509]
  We build a transparent backbone model for convolutional variational autoencoders (VAE) We propose explanation and evaluation techniques for the internal dynamics and feature relevance of prediction networks. We showcase our methods by analyzing a VAE-LSTM world model that predicts pedestrian perception in an urban traffic situation.
  arXiv  Detail & Related papers  (2024-04-26T11:57:17Z)
• LLM4Drive: A Survey of Large Language Models for Autonomous Driving [62.10344445241105]
  Large language models (LLMs) have demonstrated abilities including understanding context, logical reasoning, and generating answers. In this paper, we systematically review a research line about textitLarge Language Models for Autonomous Driving (LLM4AD).
  arXiv  Detail & Related papers  (2023-11-02T07:23:33Z)
• Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
  The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization. We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
  arXiv  Detail & Related papers  (2023-10-09T14:30:06Z)
• Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies [5.977871949434069]
  We introduce Symbolic Learning (SIL) to learn driving policies which are transparent, explainable and generalisable from available datasets. Our results demonstrate that SIL not only enhances the interpretability of driving policies but also significantly improves their applicability across varied driving situations.
  arXiv  Detail & Related papers  (2023-09-27T21:03:45Z)
• Efficient Baselines for Motion Prediction in Autonomous Driving [7.608073471097835]
  Motion Prediction (MP) of multiple surroundings agents is a crucial task in arbitrarily complex environments. We aim to develop compact models using State-Of-The-Art (SOTA) techniques for MP, including attention mechanisms and GNNs.
  arXiv  Detail & Related papers  (2023-09-06T22:18:16Z)
• Physics-informed Neural Networks-based Model Predictive Control for Multi-link Manipulators [0.0]
  We discuss nonlinear model predictive control (NMPC) for multi-body dynamics via physics-informed machine learning methods. We present the idea of enhancing PINNs by adding control actions and initial conditions as additional network inputs. We present our results using our PINN-based MPC to solve a tracking problem for a complex mechanical system.
  arXiv  Detail & Related papers  (2021-09-22T15:31:24Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)
• Out-of-Distribution Detection for Automotive Perception [58.34808836642603]
  Neural networks (NNs) are widely used for object classification in autonomous driving. NNs can fail on input data not well represented by the training dataset, known as out-of-distribution (OOD) data. This paper presents a method for determining whether inputs are OOD, which does not require OOD data during training and does not increase the computational cost of inference.
  arXiv  Detail & Related papers  (2020-11-03T01:46:35Z)
• Learning Dexterous Manipulation from Suboptimal Experts [69.8017067648129]
  Relative Entropy Q-Learning (REQ) is a simple policy algorithm that combines ideas from successful offline and conventional RL algorithms. We show how REQ is also effective for general off-policy RL, offline RL, and RL from demonstrations.
  arXiv  Detail & Related papers  (2020-10-16T18:48:49Z)
• Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning [75.56839075060819]
  Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sample-inefficient and brittle. In this work, we combine the strengths of model-based methods with the flexibility of learning-based methods to obtain a general method that is able to overcome inaccuracies in the robotics perception/actuation pipeline.
  arXiv  Detail & Related papers  (2020-05-21T19:47:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.