Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model

• URL: http://arxiv.org/abs/2601.11143v1
• Date: Fri, 16 Jan 2026 10:01:09 GMT
• Title: Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
• Abstract summary: This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with reinforcement learning (RL) on a heavy hydraulic quadruped robot.<n>The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments.
• Score: 7.275864116517311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The simulation-to-reality (sim-to-real) transfer of large-scale hydraulic robots presents a significant challenge in robotics because of the inherent slow control response and complex fluid dynamics. The complex dynamics result from the multiple interconnected cylinder structure and the difference in fluid rates of the cylinders. These characteristics complicate detailed simulation for all joints, making it unsuitable for reinforcement learning (RL) applications. In this work, we propose an analytical actuator model driven by hydraulic dynamics to represent the complicated actuators. The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments. We compare our model with neural network-based actuator models and demonstrate the advantages of our model in data-limited scenarios. The locomotion policy trained in RL with our model is deployed on a hydraulic quadruped robot, which is over 300 kg. This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with RL on a heavy hydraulic quadruped robot, demonstrating advanced sim-to-real transferability.

Related papers

• Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning [0.5277024349608834]
  Sim-to-real transfer in robot learning is often limited by discrepancies between the ideal actuator dynamics assumed during policy training and the nonlinear, hardware-dependent behavior of physical motors.<n>We introduce an alternative paradigm: actuator reality shaping.<n>We validate the approach on a single-joint high-gear-ratio servo under external loads and a 7-DOF robotic arm reaching task.
  arXiv  Detail & Related papers  (2026-07-02T14:12:41Z)
• Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks [53.12548325396181]
  Tendon drives paired with soft muscle actuation enable faster and safer robots.<n>We propose a sim-to-real pipeline that learns a neural network model of this complex actuation.<n>We successfully deploy precise goal-reaching and dynamic ball-in-a-cup policies trained entirely in simulation.
  arXiv  Detail & Related papers  (2026-04-10T16:52:54Z)
• Physics-informed Deep Mixture-of-Koopmans Vehicle Dynamics Model with Dual-branch Encoder for Distributed Electric-drive Trucks [19.506735422845182]
  We propose a fully data-driven dynamics modeling method tailored for complex distributed electric-drive trucks (DETs)<n>We first propose a novel dual-branch encoder which encodes dynamic states and provides a powerful basis for the proposed Koopman-based methods entitled KODE.<n>A physics-informed supervision mechanism is incorporated into the training process to facilitate effective learning of both the encoder and the Koopman operator.
  arXiv  Detail & Related papers  (2026-03-18T06:47:40Z)
• Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots [4.374417345150659]
  We propose a method to model the tendon forces produced by typical servo motors.<n>We then utilize our force estimation model in a GPU-accelerated tendon force-driven rigid body simulation to train RL-based controllers.<n>Our model is capable of predicting tendon forces within 3% of the maximum motor force and is robot-agnostic.
  arXiv  Detail & Related papers  (2026-03-04T18:16:58Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• Simple Models, Real Swimming: Digital Twins for Tendon-Driven Underwater Robots [8.744009526780692]
  We present a tendon-driven fish robot modeled in an efficient underwater swimmer environment.<n>Our results show that even simple, stateless models can serve as effective digital twins for soft underwater robots.
  arXiv  Detail & Related papers  (2026-02-26T17:55:22Z)
• WAKESET: A Large-Scale, High-Reynolds Number Flow Dataset for Machine Learning of Turbulent Wake Dynamics [0.7466390172678969]
  This paper introduces WAKESET, a novel, large-scale CFD dataset of highly turbulent flows.<n>The dataset captures the complex hydrodynamic interactions during the underwater recovery of an autonomous underwater vehicle.<n>It comprises 1,091 high-fidelity Reynolds-Averaged Navier-Stokes simulations, augmented to 4,364 instances, covering a wide operational envelope of speeds.
  arXiv  Detail & Related papers  (2026-02-01T18:27:10Z)
• InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation [77.07565723756119]
  We present InternVLA-A1, a vision-language-action model with dynamic predictive capabilities.<n>We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos.<n>We evaluate InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark.
  arXiv  Detail & Related papers  (2026-01-05T18:54:29Z)
• Learning based Modelling of Throttleable Engine Dynamics for Lunar Landing Mission [0.0]
  This paper presents a learning-based system identification approach for modelling of throttleable engine dynamics.<n>The developed model is validated with experimental results and used for closed-loop guidance and control simulations.
  arXiv  Detail & Related papers  (2025-11-05T16:36:52Z)
• Learning Aerodynamics for the Control of Flying Humanoid Robots [11.791887356425491]
  Flying humanoid robots face challenges in modeling and control, particularly with aerodynamic forces.<n>The technological contribution includes the mechanical design of iRonCub-Mk1, a jet-powered humanoid robot.<n>The scientific contribution offers a comprehensive approach to model and control aerodynamic forces using classical and learning techniques.
  arXiv  Detail & Related papers  (2025-05-30T23:27:44Z)
• A Supervised Machine-Learning Approach For Turboshaft Engine Dynamic Modeling Under Real Flight Conditions [0.0]
  In this work, we explore different Neural Network architectures to model the turboshaft engine of Leonardo's AW189P4 prototype.<n>The models are trained on an extensive database of real flight tests featuring a variety of operational maneuvers.<n>To complement the neural network approach, we apply Sparse Identification of Dynamics (SINDy) to derive a low-dimensional dynamical model.
  arXiv  Detail & Related papers  (2025-02-19T21:45:53Z)
• Pre-Trained Video Generative Models as World Simulators [59.546627730477454]
  We propose Dynamic World Simulation (DWS) to transform pre-trained video generative models into controllable world simulators.<n>To achieve precise alignment between conditioned actions and generated visual changes, we introduce a lightweight, universal action-conditioned module.<n> Experiments demonstrate that DWS can be versatilely applied to both diffusion and autoregressive transformer models.
  arXiv  Detail & Related papers  (2025-02-10T14:49:09Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model [18.66472547798549]
  We show how model-free reinforcement learning can be effectively used with a centroidal model to generate robust control policies for quadrupedal locomotion. We show the potential of the method by demonstrating stepping-stone locomotion, two-legged in-place balance, balance beam locomotion, and sim-to-real transfer without further adaptations.
  arXiv  Detail & Related papers  (2021-04-20T05:55:13Z)
• Populations of Spiking Neurons for Reservoir Computing: Closed Loop Control of a Compliant Quadruped [64.64924554743982]
  We present a framework for implementing central pattern generators with spiking neural networks to obtain closed loop robot control. We demonstrate the learning of predefined gait patterns, speed control and gait transition on a simulated model of a compliant quadrupedal robot.
  arXiv  Detail & Related papers  (2020-04-09T14:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.