A Q-learning approach to the continuous control problem of robot inverted pendulum balancing

• URL: http://arxiv.org/abs/2312.02649v1
• Date: Tue, 5 Dec 2023 10:40:48 GMT
• Title: A Q-learning approach to the continuous control problem of robot inverted pendulum balancing
• Authors: Mohammad Safeea, Pedro Neto
• Abstract summary: This study evaluates the application of a discrete action space reinforcement learning method (Q-learning) to the continuous control problem of robot inverted pendulum balancing. A mathematical model of the system dynamics is implemented, deduced by curve fitting on data acquired from the real system.
• Score: 0.29008108937701327
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study evaluates the application of a discrete action space reinforcement learning method (Q-learning) to the continuous control problem of robot inverted pendulum balancing. To speed up the learning process and to overcome technical difficulties related to the direct learning on the real robotic system, the learning phase is performed in simulation environment. A mathematical model of the system dynamics is implemented, deduced by curve fitting on data acquired from the real system. The proposed approach demonstrated feasible, featuring its application on a real world robot that learned to balance an inverted pendulum. This study also reinforces and demonstrates the importance of an accurate representation of the physical world in simulation to achieve a more efficient implementation of reinforcement learning algorithms in real world, even when using a discrete action space algorithm to control a continuous action.

Related papers

• Stochastic Online Optimization for Cyber-Physical and Robotic Systems [9.392372266209103]
  We propose a novel online framework for solving programming problems in the context of cyber-physical and robotic systems. Our problem formulation constraints model the evolution of a cyber-physical system, which has, in general, a continuous state and action space space is nonlinear. We show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms.
  arXiv  Detail & Related papers  (2024-04-08T09:08:59Z)
• Distributed Robust Learning based Formation Control of Mobile Robots based on Bioinspired Neural Dynamics [14.149584412213269]
  We first introduce a distributed estimator using a variable structure and cascaded design technique, eliminating the need for derivative information to improve the real time performance. Then, a kinematic tracking control method is developed utilizing a bioinspired neural dynamic-based approach aimed at providing smooth control inputs and effectively resolving the speed jump issue. To address the challenges for robots operating with completely unknown dynamics and disturbances, a learning-based robust dynamic controller is developed.
  arXiv  Detail & Related papers  (2024-03-23T04:36:12Z)
• DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control [0.0]
  Delayed Markov decision processes fulfill the Markov property by augmenting the state space of agents with a finite time window of recently committed actions. We introduce a disturbance-augmented Markov decision process in delayed settings as a novel representation to incorporate disturbance estimation in training on-policy reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2023-06-15T10:11:38Z)
• Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics [11.946807588018595]
  This paper presents a unified model-based reinforcement learning framework that bridges active exploration and uncertainty-aware deployment. The two opposing tasks of exploration and deployment are optimized through state-of-the-art sampling-based MPC. We conduct experiments on both autonomous vehicles and wheeled robots, showing promising results for both exploration and deployment.
  arXiv  Detail & Related papers  (2023-05-20T17:20:12Z)
• Exploiting Symmetry and Heuristic Demonstrations in Off-policy Reinforcement Learning for Robotic Manipulation [1.7901837062462316]
  This paper aims to define and incorporate the natural symmetry present in physical robotic environments. The proposed method is validated via two point-to-point reaching tasks of an industrial arm, with and without an obstacle. A comparison study between the proposed method and a traditional off-policy reinforcement learning algorithm indicates its advantage in learning performance and potential value for applications.
  arXiv  Detail & Related papers  (2023-04-12T11:38:01Z)
• Online Learning of Wheel Odometry Correction for Mobile Robots with Attention-based Neural Network [63.8376359764052]
  Modern robotic platforms need a reliable localization system to operate daily beside humans. Simple pose estimation algorithms based on filtered wheel and inertial odometry often fail in the presence of abrupt kinematic changes and wheel slips. We propose an innovative online learning approach for wheel odometry correction, paving the way for a robust multi-source localization system.
  arXiv  Detail & Related papers  (2023-03-21T10:30:31Z)
• Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control [49.60520501097199]
  We present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions.
  arXiv  Detail & Related papers  (2022-10-23T00:45:05Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning [70.70104870417784]
  Reinforcement learning (RL) algorithms hold the promise of enabling autonomous skill acquisition for robotic systems. In practice, real-world robotic RL typically requires time consuming data collection and frequent human intervention to reset the environment. In this work, we study how these challenges can be tackled by effective utilization of diverse offline datasets collected from previously seen tasks.
  arXiv  Detail & Related papers  (2022-07-11T08:31:22Z)
• Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning [121.9708998627352]
  Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off. This work revisits the robustness-accuracy trade-off in robot learning by analyzing if recent advances in robust training methods and theory can make adversarial training suitable for real-world robot applications.
  arXiv  Detail & Related papers  (2022-04-15T08:12:15Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.