Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion

• URL: http://arxiv.org/abs/2608.02069v1
• Date: Mon, 03 Aug 2026 11:11:13 GMT
• Title: Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion
• Abstract summary: This paper introduces Open-DiffLoco, an open-source framework for training deployable blind quadruped locomotion policies with differentiable simulation.<n>Open-DiffLoco is the first open-source framework for training deployable locomotion policies using differentiable simulation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing deployable locomotion policies through conventional reinforcement learning often requires complex reward engineering and expensive training times. While differentiable simulation offers a highly efficient alternative, open-source tools capable of end-to-end transfer of these policies to physical hardware remain limited. This paper introduces Open-DiffLoco, an open-source framework for training deployable blind quadruped locomotion policies with differentiable simulation. The framework implements the Short-Horizon Actor-Critic (SHAC) algorithm in MuJoCo XLA (MJX) and trains a proprioceptive policy that transfers to real-world hardware. The deployed policy removes privileged actor observations, including base linear velocity, and does not rely on reference trajectories. It also uses a substantially simplified reward function, enabling the robot to discover walking patterns without the complex auxiliary rewards typically used in conventional reinforcement learning pipelines. When deployed on physical hardware (a Unitree Go2 quadruped), the trained policy tracks omnidirectional velocity commands with root-mean-square error below 0.2 m/s, reaches speeds above 1 m/s, and remains robust to uneven terrain and external physical disturbances, such as lateral pushes. Across the reported configurations, training uses under 6 GB of VRAM on a single NVIDIA GeForce RTX 5080 GPU and completes in approximately 20-60 minutes. As an algorithmic extension to SHAC, we propose Jacobian-Augmented Value Estimation (JAVE), which supervises the critic Jacobians to improve early first-order policy-gradient training. To our knowledge, Open-DiffLoco is the first open-source framework for training deployable locomotion policies using differentiable simulation. Deployment videos and source code are available at: https://diffloco.martin-opat.com/

Related papers

• ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning [4.555414332509501]
  We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning framework for learning sim-to-real transferable dexterity.<n>ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior.<n>We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments.
  arXiv  Detail & Related papers  (2026-08-19T17:55:06Z)
• Patch Policy: Efficient Embodied Control via Dense Visual Representations [52.199962246592285]
  We develop Patch Policy to consume dense pre-trained patch tokens directly without the computational overhead of a full VLM.<n>Our method achieves a 40% relative improvement over policies using state-of-the-art global-pooled representations.<n>It surpasses fine-tuned OpenVLA-OFT by 18% while using roughly 0.7% of the parameters.
  arXiv  Detail & Related papers  (2026-07-20T17:59:41Z)
• CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving [66.77752555295355]
  We present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD.<n>We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-07-03T00:34:50Z)
• Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient [9.971505157889503]
  We present the decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end.<n>SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments.<n>On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards.
  arXiv  Detail & Related papers  (2026-05-26T02:35:08Z)
• Simple but Stable, Fast and Safe: Achieve End-to-end Control by High-Fidelity Differentiable Simulation [14.322775509774246]
  Obstacle avoidance is a fundamental vision-based task essential for enabling quadrotors to perform advanced applications.<n>We propose a novel end-to-end policy that directly maps depth images to low-level bodyrate commands by reinforcement learning.<n>Results demonstrate that the proposed approach achieves the highest success rate and the lowest jerk among state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-04-12T09:38:00Z)
• A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment [48.90852123901697]
  We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors.<n>Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
  arXiv  Detail & Related papers  (2025-04-21T14:25:23Z)
• AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs [68.99086112477565]
  Transformer-based large language models (LLMs) have demonstrated exceptional capabilities in sequence modeling and text generation.<n>Existing heterogeneous training methods significantly expand the scale of trainable models but introduce substantial communication overheads and CPU workloads.<n>We propose AutoHete, an automatic and efficient heterogeneous training system compatible with both single- GPU and multi- GPU environments.
  arXiv  Detail & Related papers  (2025-02-27T14:46:22Z)
• What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study [24.239835581921458]
  We investigate key factors for learning robust RL-based control policies capable of zero-shot deployment in real-world quadrotors.<n>We develop a PPO-based training framework named SimpleFlight, which integrates these five techniques.<n>We validate the efficacy of SimpleFlight on Crazyflie quadrotor, demonstrating that it achieves more than a 50% reduction in trajectory tracking error.
  arXiv  Detail & Related papers  (2024-12-16T13:31:26Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [57.278726604424556]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers.<n>Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy.<n>We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.