Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model

• URL: http://arxiv.org/abs/2601.11143v1
• Date: Fri, 16 Jan 2026 10:01:09 GMT
• Title: Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
• Authors: Minho Lee, Hyeonseok Kim, Jin Tak Kim, Sangshin Park, Jeong Hyun Lee, Jungsan Cho, Jemin Hwangbo,
• Abstract summary: This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with reinforcement learning (RL) on a heavy hydraulic quadruped robot.<n>The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments.
• Score: 7.275864116517311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The simulation-to-reality (sim-to-real) transfer of large-scale hydraulic robots presents a significant challenge in robotics because of the inherent slow control response and complex fluid dynamics. The complex dynamics result from the multiple interconnected cylinder structure and the difference in fluid rates of the cylinders. These characteristics complicate detailed simulation for all joints, making it unsuitable for reinforcement learning (RL) applications. In this work, we propose an analytical actuator model driven by hydraulic dynamics to represent the complicated actuators. The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments. We compare our model with neural network-based actuator models and demonstrate the advantages of our model in data-limited scenarios. The locomotion policy trained in RL with our model is deployed on a hydraulic quadruped robot, which is over 300 kg. This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with RL on a heavy hydraulic quadruped robot, demonstrating advanced sim-to-real transferability.

Related papers

• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• WAKESET: A Large-Scale, High-Reynolds Number Flow Dataset for Machine Learning of Turbulent Wake Dynamics [0.7466390172678969]
  This paper introduces WAKESET, a novel, large-scale CFD dataset of highly turbulent flows.<n>The dataset captures the complex hydrodynamic interactions during the underwater recovery of an autonomous underwater vehicle.<n>It comprises 1,091 high-fidelity Reynolds-Averaged Navier-Stokes simulations, augmented to 4,364 instances, covering a wide operational envelope of speeds.
  arXiv  Detail & Related papers  (2026-02-01T18:27:10Z)
• Learning based Modelling of Throttleable Engine Dynamics for Lunar Landing Mission [0.0]
  This paper presents a learning-based system identification approach for modelling of throttleable engine dynamics.<n>The developed model is validated with experimental results and used for closed-loop guidance and control simulations.
  arXiv  Detail & Related papers  (2025-11-05T16:36:52Z)
• Learning Aerodynamics for the Control of Flying Humanoid Robots [11.791887356425491]
  Flying humanoid robots face challenges in modeling and control, particularly with aerodynamic forces.<n>The technological contribution includes the mechanical design of iRonCub-Mk1, a jet-powered humanoid robot.<n>The scientific contribution offers a comprehensive approach to model and control aerodynamic forces using classical and learning techniques.
  arXiv  Detail & Related papers  (2025-05-30T23:27:44Z)
• A Supervised Machine-Learning Approach For Turboshaft Engine Dynamic Modeling Under Real Flight Conditions [0.0]
  In this work, we explore different Neural Network architectures to model the turboshaft engine of Leonardo's AW189P4 prototype.<n>The models are trained on an extensive database of real flight tests featuring a variety of operational maneuvers.<n>To complement the neural network approach, we apply Sparse Identification of Dynamics (SINDy) to derive a low-dimensional dynamical model.
  arXiv  Detail & Related papers  (2025-02-19T21:45:53Z)
• Pre-Trained Video Generative Models as World Simulators [59.546627730477454]
  We propose Dynamic World Simulation (DWS) to transform pre-trained video generative models into controllable world simulators.<n>To achieve precise alignment between conditioned actions and generated visual changes, we introduce a lightweight, universal action-conditioned module.<n> Experiments demonstrate that DWS can be versatilely applied to both diffusion and autoregressive transformer models.
  arXiv  Detail & Related papers  (2025-02-10T14:49:09Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model [18.66472547798549]
  We show how model-free reinforcement learning can be effectively used with a centroidal model to generate robust control policies for quadrupedal locomotion. We show the potential of the method by demonstrating stepping-stone locomotion, two-legged in-place balance, balance beam locomotion, and sim-to-real transfer without further adaptations.
  arXiv  Detail & Related papers  (2021-04-20T05:55:13Z)
• Populations of Spiking Neurons for Reservoir Computing: Closed Loop Control of a Compliant Quadruped [64.64924554743982]
  We present a framework for implementing central pattern generators with spiking neural networks to obtain closed loop robot control. We demonstrate the learning of predefined gait patterns, speed control and gait transition on a simulated model of a compliant quadrupedal robot.
  arXiv  Detail & Related papers  (2020-04-09T14:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.