GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning

• URL: http://arxiv.org/abs/2604.25459v1
• Date: Tue, 28 Apr 2026 10:05:39 GMT
• Title: GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning
• Abstract summary: GS-Playground is a multi-modal simulation framework designed to accelerate end-to-end perceptual learning.<n>Our system achieves a breakthrough throughput of 104 FPS at 640x480 resolution, significantly lowering the barrier for large-scale visual RL.
• Score: 55.272458304179025
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied AI research is undergoing a shift toward vision-centric perceptual paradigms. While massively parallel simulators have catalyzed breakthroughs in proprioception-based locomotion, their potential remains largely untapped for vision-informed tasks due to the prohibitive computational overhead of large-scale photorealistic rendering. Furthermore, the creation of simulation-ready 3D assets heavily relies on labor-intensive manual modeling, while the significant sim-to-real physical gap hinders the transfer of contact-rich manipulation policies. To address these bottlenecks, we propose GS-Playground, a multi-modal simulation framework designed to accelerate end-to-end perceptual learning. We develop a novel high-performance parallel physics engine, specifically designed to integrate with a batch 3D Gaussian Splatting (3DGS) rendering pipeline to ensure high-fidelity synchronization. Our system achieves a breakthrough throughput of 10^4 FPS at 640x480 resolution, significantly lowering the barrier for large-scale visual RL. Additionally, we introduce an automated Real2Sim workflow that reconstructs photorealistic, physically consistent, and memory-efficient environments, streamlining the generation of complex simulation-ready scenes. Extensive experiments on locomotion, navigation, and manipulation demonstrate that GS-Playground effectively bridges the perceptual and physical gaps across diverse embodied tasks. Project homepage: https://gsplayground.github.io.

Related papers

• Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator [70.34667754804168]
  Embodied navigation aims to build agents that interpret multimodal goals, reason in 3D space, and reach target destinations reliably in the real world.<n>We introduce Image2Sim, a real-time neural simulation framework that constructs high-quality interactive environments from posed RGB-D image sequences.
  arXiv  Detail & Related papers  (2026-07-07T02:42:41Z)
• GASE: Gaussian Splatting-Based Automated System for Reconstructing Embodied-Simulation Environments [63.222920701596564]
  Training embodied agents in the real world requires skilled operators and expensive hardware.<n> Simulation environments offer a compelling alternative by enabling large-scale, cost-effective data augmentation.<n>Fastly constructing high-fidelity simulation scenes with a minimal sim-to-real gap has become a critical objective in robot learning.
  arXiv  Detail & Related papers  (2026-06-16T05:00:42Z)
• V-Dreamer: Automating Robotic Simulation and Trajectory Synthesis via Video Generation Priors [30.872127145725187]
  V-Dreamer is a fully automated framework that generates open-vocabulary, simulation-ready manipulation environments.<n>V-Dreamer employs a novel generative pipeline that constructs physically grounded 3D scenes.<n>We show that our policies robustly generalize to unseen objects in simulation and achieve effective sim-to-real transfer.
  arXiv  Detail & Related papers  (2026-03-19T12:00:04Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• GaussGym: An open-source real-to-sim framework for learning locomotion from pixels [78.05453137978132]
  We present a novel approach for photorealistic robot simulation that integrates 3D Gaussian Splatting as a drop-in within vectorized physics simulators.<n>This enables unprecedented speed -- exceeding 100,000 steps per second on consumer GPU.<n>We additionally demonstrate its applicability in a sim-to-real robotics setting.
  arXiv  Detail & Related papers  (2025-10-17T06:34:52Z)
• VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion [25.440573256776133]
  This paper presents a Real-to-Sim-to-Real framework that generates and physically interactive "digital twin" simulation environments for visual navigation and locomotion learning.
  arXiv  Detail & Related papers  (2025-02-03T17:15:05Z)
• Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation [62.5805866419814]
  Vid2Sim is a novel framework that bridges the sim2real gap through a scalable and cost-efficient real2sim pipeline for neural 3D scene reconstruction and simulation.<n>Experiments demonstrate that Vid2Sim significantly improves the performance of urban navigation in the digital twins and real world by 31.2% and 68.3% in success rate.
  arXiv  Detail & Related papers  (2025-01-12T03:01:15Z)
• VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality [39.53150683721031]
  Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction. The components of our Virtual Reality system are designed for high efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-01-30T01:28:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.