Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem

• URL: http://arxiv.org/abs/2510.08768v1
• Date: Thu, 09 Oct 2025 19:36:18 GMT
• Title: Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem
• Abstract summary: Reinforcement learning policies often fail to generalize to new robots, tasks, or environments with different physical parameters.<n>This paper presents a simple, zero-shot transfer method based on Buckingham's Pi Theorem to address this limitation.
• Score: 42.37643072381109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) policies often fail to generalize to new robots, tasks, or environments with different physical parameters, a challenge that limits their real-world applicability. This paper presents a simple, zero-shot transfer method based on Buckingham's Pi Theorem to address this limitation. The method adapts a pre-trained policy to new system contexts by scaling its inputs (observations) and outputs (actions) through a dimensionless space, requiring no retraining. The approach is evaluated against a naive transfer baseline across three environments of increasing complexity: a simulated pendulum, a physical pendulum for sim-to-real validation, and the high-dimensional HalfCheetah. Results demonstrate that the scaled transfer exhibits no loss of performance on dynamically similar contexts. Furthermore, on non-similar contexts, the scaled policy consistently outperforms the naive transfer, significantly expanding the volume of contexts where the original policy remains effective. These findings demonstrate that dimensional analysis provides a powerful and practical tool to enhance the robustness and generalization of RL policies.

Related papers

• Robust Data-Collection Policy Learning for Low-Variance Online Policy Evaluation [28.722908721336992]
  We propose a double-loop gradient-based algorithm for learning behavior policies that are both efficient and robust to transition uncertainty.<n>We demonstrate that our method is less sensitive to transition perturbations than existing approaches.
  arXiv  Detail & Related papers  (2026-08-25T07:10:06Z)
• A Single Diffusion-Policy Controller for Multi-Task Block Pushing with Zero-Shot Sim-to-Real Transfer [15.906272895307664]
  We explore training diffusion policies from scratch using reinforcement learning (RL) for multi-task robotic manipulation.<n>We aim to train a single diffusion policy for block-pushing tasks with multiple shapes.<n>We evaluate whether the trained diffusion policy transfers in zero-shot to real-world tasks under varying environmental conditions.
  arXiv  Detail & Related papers  (2026-07-12T19:48:46Z)
• Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning [50.738952715864116]
  Expressive continuous control policies form the backbone of advances in scaling imitation learning for simulated and real robot control.<n>We propose QGF (Q-Guided Flow), an RL algorithm that performs policy optimization entirely at test time.<n> Empirically, QGF outperforms prior test-time RL methods on single-task and goal-conditioned offline RL benchmarks.
  arXiv  Detail & Related papers  (2026-06-09T16:45:57Z)
• Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation [51.463992799813816]
  A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data.<n>We introduce Center-of-Pressure, an effective tactile representation grounded in physical principles.<n>We evaluate CoP on two blind, challenging contact-rich manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-27T17:59:02Z)
• End-to-end example-based sim-to-real RL policy transfer based on neural stylisation with application to robotic cutting [3.398964351541323]
  We propose a novel method for sim-to-real transfer of reinforcement learning policies.<n>We employ a variational autoencoder to jointly learn self-supervised feature representations for style transfer.<n>We demonstrate the application of our approach based on the case study of robot cutting of unknown materials.
  arXiv  Detail & Related papers  (2026-01-28T18:45:55Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [86.99017195607077]
  We address real-time sampling and estimation of autoregressive Markovian sources in wireless networks.<n>We propose a graphical reinforcement learning framework for policy optimization.<n>Theoretically, our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs.
  arXiv  Detail & Related papers  (2026-01-19T02:18:45Z)
• Balance Equation-based Distributionally Robust Offline Imitation Learning [8.607736795429638]
  Imitation Learning (IL) has proven highly effective for robotic and control tasks where manually designing reward functions or explicit controllers is infeasible.<n>Standard IL methods implicitly assume that the environment dynamics remain fixed between training and deployment.<n>We address this challenge through Balance Equation-based Distributionally Robust Offline Learning.<n>We formulate the problem as a distributionally robust optimization over an uncertainty set of transition models, seeking a policy that minimizes the imitation loss under the worst-case transition distribution.
  arXiv  Detail & Related papers  (2025-11-11T07:48:09Z)
• Dynamic Manipulation of Deformable Objects in 3D: Simulation, Benchmark and Learning Strategy [88.8665000676562]
  Prior methods often simplify the problem to low-speed or 2D settings, limiting their applicability to real-world 3D tasks.<n>To mitigate data scarcity, we introduce a novel simulation framework and benchmark grounded in reduced-order dynamics.<n>We propose Dynamics Informed Diffusion Policy (DIDP), a framework that integrates imitation pretraining with physics-informed test-time adaptation.
  arXiv  Detail & Related papers  (2025-05-23T03:28:25Z)
• Maximum Total Correlation Reinforcement Learning [23.209609715886454]
  We introduce a modification of the reinforcement learning problem that additionally maximizes the total correlation within the induced trajectories.<n>In simulated robot environments, our method naturally generates policies that induce periodic and compressible trajectories.
  arXiv  Detail & Related papers  (2025-05-22T14:48:00Z)
• Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Picking [5.027571997864706]
  This paper addresses the post-convergence sim-to-real transfer problem by introducing a worst-case performance transference optimization approach.<n>Experiments demonstrate its effectiveness in transferring RL-based locomotion policies from simulation to real-world laboratory tests.
  arXiv  Detail & Related papers  (2025-04-21T19:48:05Z)
• Learning Optimal Deterministic Policies with Stochastic Policy Gradients [62.81324245896716]
  Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. In common practice, convergence (hyper)policies are learned only to deploy their deterministic version. We show how to tune the exploration level used for learning to optimize the trade-off between the sample complexity and the performance of the deployed deterministic policy.
  arXiv  Detail & Related papers  (2024-05-03T16:45:15Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [86.99017195607077]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.<n>Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)
• Towards Robust Bisimulation Metric Learning [3.42658286826597]
  Bisimulation metrics offer one solution to representation learning problem. We generalize value function approximation bounds for on-policy bisimulation metrics to non-optimal policies. We find that these issues stem from an underconstrained dynamics model and an unstable dependence of the embedding norm on the reward signal.
  arXiv  Detail & Related papers  (2021-10-27T00:32:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.