UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms

• URL: http://arxiv.org/abs/2605.30313v3
• Date: Tue, 02 Jun 2026 16:21:55 GMT
• Title: UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms
• Abstract summary: In simulation-dominated robot control, the essential question is not which processor runs physics, but whether simulation throughput, policy learning, and runtime synchronization form an efficient end-to-end loop.<n>We present UniLab, a heterogeneous CPU-resident / GPU-learning architecture that decouples CPU-parallel simulation from GPU policy updates through a unified runtime for data movement, buffering, and synchronization.
• Score: 68.40808444348735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulation-based RL for contemporary robot control is increasingly organized around GPU-resident simulation: physics, rollout collection, and learning are placed on a single GPU-centric execution path. This paradigm has greatly improved training speed, but it has also encouraged a default assumption that efficient training requires physics to reside on the GPU. We revisit this assumption. Our view is that, in simulation-dominated robot control, the essential question is not which processor runs physics, but whether simulation throughput, policy learning, and runtime synchronization form an efficient end-to-end loop. We present UniLab, a heterogeneous CPU-simulation / GPU-learning architecture that decouples CPU-parallel simulation from GPU policy updates through a unified runtime for data movement, buffering, and synchronization. UniLab is implemented as a complete and extensible training system using MuJoCoUni and MotrixSim CPU-batched physics backends, supporting PPO, FastSAC, FlashSAC, and APPO. On representative simulation-based robot control tasks, UniLab improves end-to-end training efficiency by 3--10$\times$ under the same hardware configuration, while reducing dependence on the NVIDIA CUDA-based software stack and supporting cross-platform execution on the Apple macOS platform and the AMD ROCm and Intel XPU accelerator backends. These results show that GPU simulation is an effective path to efficient training, but not a necessary one, broadening the practical system choices available for robot RL training. Project page: https://unilabsim.github.io.

Related papers

• InstantMimic: A High Performance System for Learning Physics-based Skills in Seconds [4.282159812965446]
  We present InstantMimic, a system that makes the entire training loop GPU-native.<n>Our unified pipeline integrates simulation, environment, policy inference, and policy updates within a single execution flow.<n>As a result, InstantMimic reduces training time for diverse physics-based skills to a few seconds.
  arXiv  Detail & Related papers  (2026-09-09T07:23:51Z)
• Multi-GPU Quantum Circuit Simulation and the Impact of Network Performance [0.7340017786387767]
  We present the introduction of MPI into the QED-C Application-Oriented Benchmarks to facilitate benchmarking on HPC systems.<n>We benchmark using a variety of interconnect paths, including the recent NVIDIA Grace Blackwell NVL72 architecture.<n>We show that while improvements to GPU architecture have led to speedups of over 4.5X, advances in interconnect performance have had a larger impact with over 16X performance improvements in time to solution.
  arXiv  Detail & Related papers  (2025-11-18T17:04:28Z)
• Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning [72.43357471969564]
  Isaac Lab combines high-fidelity GPU parallel physics, rendering, and a modular, composable architecture for designing environments and training robot policies.<n>We highlight its application to a diverse set of challenges, including whole-body control, cross-embodiment mobility, contact-rich and dexterous manipulation, and the integration of human demonstrations for skill acquisition.<n>We believe Isaac Lab's combination of advanced simulation capabilities, rich sensing, and data-center scale execution will help unlock the next generation of breakthroughs in robotics research.
  arXiv  Detail & Related papers  (2025-11-06T21:43:02Z)
• ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI [26.070510680936486]
  ManiSkill3 is the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation.<n>ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more.
  arXiv  Detail & Related papers  (2024-10-01T06:10:39Z)
• In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD [51.04126395480625]
  Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. This work offers a solution by simplifying this coupling and enabling in situ training and inference on heterogeneous clusters.
  arXiv  Detail & Related papers  (2023-06-22T14:07:54Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• Optimizing Data Collection in Deep Reinforcement Learning [4.9709347068704455]
  GPU vectorization can achieve up to $1024times$ speedup over commonly used CPU simulators. We show that simulator kernel fusion speedups with a simple simulator are $11.3times$ and increase by up to $1024times$ as simulator complexity increases in terms of memory bandwidth requirements.
  arXiv  Detail & Related papers  (2022-07-15T20:22:31Z)
• ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning [141.58588761593955]
  We present a library ElegantRL-podracer for cloud-native deep reinforcement learning. It efficiently supports millions of cores to carry out massively parallel training at multiple levels. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU cores in a single GPU.
  arXiv  Detail & Related papers  (2021-12-11T06:31:21Z)
• Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning [5.786157611032947]
  Isaac Gym offers a high performance learning platform to train policies for wide variety of robotics tasks on GPU. Both physics simulation and the neural network policy training training training communicate by directly passing data directly to PyTorch. This leads to blazing fast training times for complex robotics tasks on a single GPU with 2-3 orders of magnitude improvements.
  arXiv  Detail & Related papers  (2021-08-24T01:38:11Z)
• Megaverse: Simulating Embodied Agents at One Million Experiences per Second [75.1191260838366]
  We present Megaverse, a new 3D simulation platform for reinforcement learning and embodied AI research. Megaverse is up to 70x faster than DeepMind Lab in fully-shaded 3D scenes with interactive objects. We use Megaverse to build a new benchmark that consists of several single-agent and multi-agent tasks.
  arXiv  Detail & Related papers  (2021-07-17T03:16:25Z)
• Large Batch Simulation for Deep Reinforcement Learning [101.01408262583378]
  We accelerate deep reinforcement learning-based training in visually complex 3D environments by two orders of magnitude over prior work. We realize end-to-end training speeds of over 19,000 frames of experience per second on a single and up to 72,000 frames per second on a single eight- GPU machine. By combining batch simulation and performance optimizations, we demonstrate that Point navigation agents can be trained in complex 3D environments on a single GPU in 1.5 days to 97% of the accuracy of agents trained on a prior state-of-the-art system.
  arXiv  Detail & Related papers  (2021-03-12T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.