SR-Platform: An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis

• URL: http://arxiv.org/abs/2605.14700v1
• Date: Thu, 14 May 2026 11:14:20 GMT
• Title: SR-Platform: An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis
• Abstract summary: SR-Platform is a production-deployed agentic system that converts free-form natural language descriptions into executable MuJoCo environments.<n>It decomposes scene synthesis into four stages: an orchestrator that converts user intent into a structured scene plan; an asset forge that retrieves cached assets or generates new 3D geometry.<n>The system is deployed as a nine-service Docker stack with WebSocket progress streaming, MinIO-backed mesh storage, Qdrant-based semantic asset retrieval, Redis job state, and InfluxDB telemetry.
• Score: 0.3499870393443268
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating robot simulation environments remains a major bottleneck in simulation-based robot learning. Constructing a training-ready MuJoCo scene typically requires expertise in 3D asset modeling, MJCF specification, spatial layout, collision avoidance, and robot-model integration. We present SR-Platform, a production-deployed agentic system that converts free-form natural language descriptions into executable, physically valid MuJoCo environments. SR-Platform decomposes scene synthesis into four stages: an LLM-based orchestrator that converts user intent into a structured scene plan; an asset forge that retrieves cached assets or generates new 3D geometry through LLM-to-CadQuery synthesis; a layout architect that assigns object poses and verifies industrial constraints; and a bridge layer that assembles the final MJCF scene and merges the selected robot model. The system is deployed as a nine-service Docker stack with WebSocket progress streaming, MinIO-backed mesh storage, Qdrant-based semantic asset retrieval, Redis job state, and InfluxDB telemetry. Using 30 days of production telemetry covering 611 successful LLM calls, SR-Platform generates five-object scenes with a median end-to-end latency of approximately 50 s, while cache-accelerated scenes complete in approximately 30-40 s. The asset forge shows an 11.3% first-attempt retry rate with automatic recovery, and cached asset retrieval removes per-object LLM calls for previously generated object types. These results show that agentic scene synthesis can reduce the manual effort required to create diverse robot training environments, enabling users to produce executable MuJoCo scenes from plain English prompts in under one minute.

Related papers

• RoboCousin: Build Your Own Simulation Playground for Robust Bimanual Robotic Manipulation [16.581929671943325]
  Bimanual manipulation policies require large and diverse training datasets.<n>Existing pipelines typically operate within closed asset libraries and predefined scenes.<n>We present RoboCousin, a simulation-based data-generation platform.
  arXiv  Detail & Related papers  (2026-09-08T07:13:07Z)
• Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling [11.305288161442995]
  Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed.<n>Lucia parses the video into a scene graph whose nodes carry per-instance multi-view evidence.<n>Lucia places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction.
  arXiv  Detail & Related papers  (2026-08-31T14:03:20Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects [69.20984454755512]
  Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation.<n>Existing pipelines represent generated content as static meshes and inherit articulation only from curated asset libraries.<n>We present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world.
  arXiv  Detail & Related papers  (2026-05-19T09:31:04Z)
• Articraft: An Agentic System for Scalable Articulated 3D Asset Generation [79.1741181153232]
  A bottleneck in learning to understand articulated 3D objects is the lack of large and diverse datasets.<n>We propose to leverage large language models (LLMs) to generate articulated assets at scale.<n>We then introduce a new agentic system, Articraft, that writes such programs automatically.
  arXiv  Detail & Related papers  (2026-05-14T17:59:18Z)
• Agentic Video Generation: From Text to Executable Event Graphs via Tool-Constrained LLM Planning [4.964902130083661]
  Existing multi-agent video generation systems use LLM agents to orchestrate neural video generators.<n>We present an agentic system that inverts this paradigm, constructing a formal Graph of Events in Space and Time.<n>A programmatic state backend enforces all simulator constraints through validated tool calls, guaranteeing that every generated specification is executable.
  arXiv  Detail & Related papers  (2026-04-11T23:51:13Z)
• SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLM [14.374484080984423]
  We propose SIMART, a unified MLLM framework that jointly performs part-level decomposition and kinematic prediction.<n>By introducing a Sparse 3D VQ-VAE, SIMART reduces token counts by 70% vs. dense voxel tokens, enabling high-fidelity multi-part assemblies.
  arXiv  Detail & Related papers  (2026-03-24T16:16:52Z)
• SAGE: Scalable Agentic 3D Scene Generation for Embodied AI [67.43935343696982]
  Existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes.<n>We present SAGE, an agentic framework that, given a user-specified embodied task, understands the intent and automatically generates simulation-ready environments at scale.<n>The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training.
  arXiv  Detail & Related papers  (2026-02-10T18:59:55Z)
• SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes [19.995619927680476]
  SceneSmith builds environments from architectural layout to natural furniture population.<n>SceneSmith generates more objects than prior methods, with 2% interobject collisions and 96% of objects remaining stable under physics simulation.<n>SceneSmith environments can be used in an end-to-end pipeline for automatic policy evaluation.
  arXiv  Detail & Related papers  (2026-02-09T19:56:04Z)
• EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis [101.67583081810136]
  Large language models (LLMs) are expected to be trained to act as agents in various real-world environments.<n>This process relies on rich and varied tool-interaction sandboxes.<n>We propose EnvScaler, an automated framework for scalable tool-interaction environments.
  arXiv  Detail & Related papers  (2026-01-09T14:32:06Z)
• URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Model [76.08429266631823]
  We propose an end-to-end automatic reconstruction framework based on a 3D multimodal large language model (MLLM)<n>URDF-Anything utilizes an autoregressive prediction framework based on point-cloud and text multimodal input to jointly optimize geometric segmentation and kinematic parameter prediction.<n> Experiments on both simulated and real-world datasets demonstrate that our method significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2025-11-02T13:45:51Z)
• Steerable Scene Generation with Post Training and Inference-Time Search [21.854690970995648]
  Training robots in simulation requires diverse 3D scenes that reflect specific challenges of downstream tasks.<n>We generate large-scale scene data using procedural models that approximate realistic environments for robotic manipulation.<n>We release a dataset of over 44 million SE(3) scenes spanning five diverse environments.
  arXiv  Detail & Related papers  (2025-05-07T22:07:42Z)
• DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution [114.61347672265076]
  Development of MLLMs for real-world robots is challenging due to the typically limited computation and memory capacities available on robotic platforms. We propose a Dynamic Early-Exit Framework for Robotic Vision-Language-Action Model (DeeR) that automatically adjusts the size of the activated MLLM. DeeR demonstrates significant reductions in computational costs of LLM by 5.2-6.5x and GPU memory of LLM by 2-6x without compromising performance.
  arXiv  Detail & Related papers  (2024-11-04T18:26:08Z)
• SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code [76.22337677728109]
  SceneCraft is a Large Language Model (LLM) Agent converting text descriptions into Blender-executable Python scripts. SceneCraft renders complex scenes with up to a hundred 3D assets. We tackle these challenges through a combination of advanced abstraction, strategic planning, and library learning.
  arXiv  Detail & Related papers  (2024-03-02T16:16:26Z)
• Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks [54.60571399091711]
  Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. We present an interactive planning technique for partially observable tasks using LLMs.
  arXiv  Detail & Related papers  (2023-12-11T22:54:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.