Low-cost Real-world Implementation of the Swing-up Pendulum for Deep Reinforcement Learning Experiments

• URL: http://arxiv.org/abs/2503.11065v1
• Date: Fri, 14 Mar 2025 04:18:36 GMT
• Title: Low-cost Real-world Implementation of the Swing-up Pendulum for Deep Reinforcement Learning Experiments
• Authors: Peter Böhm, Pauline Pounds, Archie C. Chapman,
• Abstract summary: We describe a low-cost physical inverted pendulum apparatus and software environment for exploring sim-to-real DRL methods.<n>In particular, the design of our apparatus enables detailed examination of the delays that arise in physical systems when sensing, communicating, learning, inferring and actuating.<n>Our design shows how commercial, off-the-shelf electronics and electromechanical and sensor systems, combined with common metal extrusions, dowel and 3D printed couplings provide a pathway for affordable physical DRL apparatus.
• Score: 4.669957449088593
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep reinforcement learning (DRL) has had success in virtual and simulated domains, but due to key differences between simulated and real-world environments, DRL-trained policies have had limited success in real-world applications. To assist researchers to bridge the \textit{sim-to-real gap}, in this paper, we describe a low-cost physical inverted pendulum apparatus and software environment for exploring sim-to-real DRL methods. In particular, the design of our apparatus enables detailed examination of the delays that arise in physical systems when sensing, communicating, learning, inferring and actuating. Moreover, we wish to improve access to educational systems, so our apparatus uses readily available materials and parts to reduce cost and logistical barriers. Our design shows how commercial, off-the-shelf electronics and electromechanical and sensor systems, combined with common metal extrusions, dowel and 3D printed couplings provide a pathway for affordable physical DRL apparatus. The physical apparatus is complemented with a simulated environment implemented using a high-fidelity physics engine and OpenAI Gym interface.

Related papers

• A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment [48.90852123901697]
  We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors. Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
  arXiv  Detail & Related papers  (2025-04-21T14:25:23Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning [7.714620721734689]
  "Box o Flows" is an experimental control system for systematically evaluating RL algorithms in dynamic real-world scenarios. We show how state-of-the-art model-free RL algorithms can synthesize a variety of complex behaviors via simple reward specifications. We believe that the insights gained from this preliminary study and the availability of systems like the Box o Flows support the way forward for developing systematic RL algorithms.
  arXiv  Detail & Related papers  (2024-02-08T23:35:03Z)
• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning [85.21378553454672]
  We develop a library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment. We find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent robustness recovery and correction behaviors.
  arXiv  Detail & Related papers  (2024-01-29T10:01:10Z)
• Random resistive memory-based deep extreme point learning machine for unified visual processing [67.51600474104171]
  We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM) Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
  arXiv  Detail & Related papers  (2023-12-14T09:46:16Z)
• DRIFT: Deep Reinforcement Learning for Intelligent Floating Platforms Trajectories [18.420795137038677]
  Floating platforms serve as versatile test-beds to emulate micro-gravity environments on Earth. Our suite achieves robustness, adaptability, and good transferability from simulation to reality.
  arXiv  Detail & Related papers  (2023-10-06T14:11:35Z)
• Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement Learning [21.79206567364126]
  Millimeter-wave (mmWave) communication is a vital component of future mobile networks, making it ideal for indoor navigation in complex environments. Traditional physics-based methods, such as following the angle of arrival (AoA), often fall short in complex scenarios. We propose a Physics-Informed Reinforcement Learning (PIRL) approach that leverages the physical insights provided by digital twins to shape the reinforcement learning (RL) reward function.
  arXiv  Detail & Related papers  (2023-06-11T20:33:22Z)
• SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning [78.76052604441519]
  SurRoL is an RL-centered simulation platform for surgical robot learning compatible with the da Vinci Research Kit (dVRK) Ten learning-based surgical tasks are built in the platform, which are common in the real autonomous surgical execution. We evaluate SurRoL using RL algorithms in simulation, provide in-depth analysis, deploy the trained policies on the real dVRK, and show that our SurRoL achieves better transferability in the real world.
  arXiv  Detail & Related papers  (2021-08-30T07:43:47Z)
• RL STaR Platform: Reinforcement Learning for Simulation based Training of Robots [3.249853429482705]
  Reinforcement learning (RL) is a promising field to enhance robotic autonomy and decision making capabilities for space robotics. This paper introduces the RL STaR platform, and how researchers can use it through a demonstration.
  arXiv  Detail & Related papers  (2020-09-21T03:09:53Z)
• RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real [74.45688231140689]
  We introduce the RL-scene consistency loss for image translation, which ensures that the translation operation is invariant with respect to the Q-values associated with the image. We obtain RL-CycleGAN, a new approach for simulation-to-real-world transfer for reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-16T08:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.