GROVE: Grounded Pedestrian Simulation via Natural Language for Interactive Social Robot Navigation

• URL: http://arxiv.org/abs/2606.25504v1
• Date: Wed, 24 Jun 2026 07:39:01 GMT
• Title: GROVE: Grounded Pedestrian Simulation via Natural Language for Interactive Social Robot Navigation
• Abstract summary: Pedestrian simulation is a critical component for training and deploying social robot navigation approaches.<n>GROVE is a text-to-scenario pedestrian simulation framework that combines state-of-the-art approaches to produce realistic, socially challenging scenarios.<n>The human simulation is directly integrated into Isaac Sim, Gazebo, and RViz simulators for robot deployment in highly social environments.
• Score: 3.066376644627784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pedestrian simulation is a critical component for training and deploying social robot navigation approaches, yet it remains a largely rigid system that repeatedly requires manual data generation to define even simple scenarios. We propose GROVE, a text-to-scenario pedestrian simulation framework that combines state-of-the-art approaches to produce realistic, socially challenging scenarios for social robot navigation. Our framework allows users to customize one of several common presets (emergency, queuing, normal) or even enter a fully independent prompt to generate a highly customizable pedestrian simulation. Multiple modules separately ensure the realism and soundness of long-horizon human behavior, medium-horizon pedestrian navigation, and short-horizon robot/social interactions. Each module is tuned by the prompt in a way that reflects the user intent across all aspects of pedestrian simulation. By dynamically selecting one of several state-of-the-art (SotA) approaches in our modules based on the scenario, we capture many situational nuances of pedestrian behavior in order to narrow the simulation-to-real (sim2real) gap. The human simulation is directly integrated into Isaac Sim, Gazebo, and RViz simulators for robot deployment in highly social environments. We validate our approach through qualitative comparison against existing pedestrian simulation baselines across scenarios of varying complexity in residential, hospital, and office environments. The result is a high-fidelity pedestrian simulation that challenges social robot navigation with complex, diverse, realistic human behaviors.

Related papers

• IR-SIM: A Lightweight Skill-Native Simulator for Navigation, Learning, and Benchmarking [59.6657632820611]
  IR-SIM is a skill-native navigation simulator designed for rapid scenario construction, benchmarking, and robot learning.<n> scenarios are entirely defined by YAML configuration files that specify mobile robot kinematics, geometric collision checking, LiDAR sensing, visualization, and behavior modules.<n>The experiments showcase the convenience and versatility of IR-SIM in multiple tasks.
  arXiv  Detail & Related papers  (2026-06-07T16:55:37Z)
• ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation [90.4702774169675]
  We propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation.<n>Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data.<n>We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments.
  arXiv  Detail & Related papers  (2026-04-13T12:25:45Z)
• PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies [88.78188489161028]
  We introduce Policy Evaluation and Environment Reconstruction in Simulation (PolaRiS)<n>PolaRiS is a scalable real-to-sim framework for high-fidelity simulated robot evaluation.<n>We show that PolaRiS evaluations provide a much stronger correlation to real world generalist policy performance than existing simulated benchmarks.
  arXiv  Detail & Related papers  (2025-12-18T18:49:41Z)
• Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions [27.247431258140463]
  We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world videos.<n>We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing.
  arXiv  Detail & Related papers  (2025-11-06T18:52:08Z)
• The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio [138.07247714782412]
  MultiGen is a framework that integrates large-scale generative models into traditional physics simulators.<n>We demonstrate effective zero-shot transfer to real-world pouring with novel containers and liquids.
  arXiv  Detail & Related papers  (2025-07-03T17:59:58Z)
• YuLan-OneSim: Towards the Next Generation of Social Simulator with Large Language Models [50.35333054932747]
  We introduce a novel social simulator called YuLan-OneSim.<n>Users can simply describe and refine their simulation scenarios through natural language interactions with our simulator.<n>We implement 50 default simulation scenarios spanning 8 domains, including economics, sociology, politics, psychology, organization, demographics, law, and communication.
  arXiv  Detail & Related papers  (2025-05-12T14:05:17Z)
• Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin [8.498460043101499]
  We introduce real-is-sim, a new approach to integrating simulation into behavior cloning pipelines.<n>In contrast to real-only methods, which lack the ability to safely test policies before deployment, and sim-to-real methods, which require complex adaptation to cross the sim-to-real gap.<n>Our framework allows policies to seamlessly switch between running on real hardware and running in parallelized virtual environments.
  arXiv  Detail & Related papers  (2025-04-04T17:05:56Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.