Learning Control from Raw Position Measurements

• URL: http://arxiv.org/abs/2301.13183v1
• Date: Mon, 30 Jan 2023 18:50:37 GMT
• Title: Learning Control from Raw Position Measurements
• Authors: Fabio Amadio, Alberto Dalla Libera, Daniel Nikovski, Ruggero Carli, Diego Romeres
• Abstract summary: We propose a Model-Based Reinforcement Learning (MBRL) algorithm named VF-MC-PILCO. It is specifically designed for application to mechanical systems where velocities cannot be directly measured.
• Score: 13.79048931313603
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a Model-Based Reinforcement Learning (MBRL) algorithm named VF-MC-PILCO, specifically designed for application to mechanical systems where velocities cannot be directly measured. This circumstance, if not adequately considered, can compromise the success of MBRL approaches. To cope with this problem, we define a velocity-free state formulation which consists of the collection of past positions and inputs. Then, VF-MC-PILCO uses Gaussian Process Regression to model the dynamics of the velocity-free state and optimizes the control policy through a particle-based policy gradient approach. We compare VF-MC-PILCO with our previous MBRL algorithm, MC-PILCO4PMS, which handles the lack of direct velocity measurements by modeling the presence of velocity estimators. Results on both simulated (cart-pole and UR5 robot) and real mechanical systems (Furuta pendulum and a ball-and-plate rig) show that the two algorithms achieve similar results. Conveniently, VF-MC-PILCO does not require the design and implementation of state estimators, which can be a challenging and time-consuming activity to be performed by an expert user.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation [41.38091115195305]
  This work presents the experimental implementation of a deep neural network (DNN) based nonlinear MPC for Homogeneous Charge Compression Ignition (HCCI) combustion control. Using the acados software package to enable the real-time implementation of the MPC on an ARM Cortex A72, the optimization calculations are completed within 1.4 ms. The IMEP trajectory following of the developed controller was excellent, with a root-mean-square error of 0.133 bar, in addition to observing process constraints.
  arXiv  Detail & Related papers  (2023-10-12T15:03:50Z)
• Multirotor Ensemble Model Predictive Control I: Simulation Experiments [0.0]
  An ensemble-represented Gaussian process performs the backward calculations to determine optimal gains for the initial time. We construct the EMPC for terminal control and regulation problems and apply it to the control of a simulated, identical-twin study.
  arXiv  Detail & Related papers  (2023-05-22T01:32:17Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application [12.854118767247453]
  We present a Model-Based Reinforcement Learning (MBRL) algorithm named emphMonte Carlo Probabilistic Inference for Learning COntrol (MC-PILCO) The algorithm relies on Gaussian Processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. Numerical comparisons in a simulated cart-pole environment show that MC-PILCO exhibits better data efficiency and control performance.
  arXiv  Detail & Related papers  (2021-01-28T17:01:15Z)
• Model-based Policy Search for Partially Measurable Systems [9.335154302282751]
  We propose a Model-Based Reinforcement Learning (MBRL) algorithm for Partially Measurable Systems (PMS) The proposed algorithm, named Monte Carlo Probabilistic Inference for Learning COntrol for Partially Measurable Systems (MC-PILCO4PMS), relies on Gaussian Processes (GPs) to model the system dynamics. The effectiveness of the proposed algorithm has been tested both in simulation and in two real systems.
  arXiv  Detail & Related papers  (2021-01-21T17:39:22Z)
• Fast and differentiable simulation of driven quantum systems [58.720142291102135]
  We introduce a semi-analytic method based on the Dyson expansion that allows us to time-evolve driven quantum systems much faster than standard numerical methods. We show results of the optimization of a two-qubit gate using transmon qubits in the circuit QED architecture.
  arXiv  Detail & Related papers  (2020-12-16T21:43:38Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Model-Based Reinforcement Learning for Physical Systems Without Velocity and Acceleration Measurements [19.060544153434428]
  We propose a derivative-free model learning framework for Reinforcement Learning (RL) algorithms based on Gaussian Process Regression (GPR) In many mechanical systems, only positions can be measured by the sensing instruments. Tests performed on two real platforms show that the considered state definition combined with the proposed model improves estimation performance.
  arXiv  Detail & Related papers  (2020-02-25T01:58:34Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.