SONG: A Photorealistic 3D Gaussian Simulation Platform for Benchmarking Social Navigation
- URL: http://arxiv.org/abs/2607.25219v1
- Date: Tue, 28 Jul 2026 02:51:24 GMT
- Title: SONG: A Photorealistic 3D Gaussian Simulation Platform for Benchmarking Social Navigation
- Abstract summary: We introduce SONG, a SOcial Navigation platform powered by 3D Gaussian splatting (3DGS)<n>It drives pedestrians using semantically grounded trajectories generated by a large language model, and synthesizes their full-body motion with a trajectory-conditioned generator.<n>On top of the platform, we curate SONG-Bench, a set of evaluation episodes stratified by difficulty, and propose a multi-dimensional metric suite covering effectiveness, safety, and social compliance.
- Score: 32.200742569755405
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Social navigation has progressed from simplified 2D environments toward a more general vision-based setting, in which a robot needs to achieve socially compliant behavior purely from onboard visual observations. Yet supporting simulation platforms have not kept pace: existing options either lack visual observations, lack moving human avatars, or fall short of real-world fidelity in appearance and pedestrian behavior, offering limited support for advancing vision-based social navigation. We introduce SONG, a SOcial Navigation platform powered by 3D Gaussian splatting (3DGS). It leverages 3DGS for both scene and avatar representations, drives pedestrians using semantically grounded trajectories generated by a large language model, and synthesizes their full-body motion with a trajectory-conditioned generator to produce continuous, natural movement. On top of the platform, we curate SONG-Bench, a set of evaluation episodes stratified by difficulty, and propose a multi-dimensional metric suite covering effectiveness, safety, and social compliance. A systematic evaluation of representative navigation baselines reveals three findings: (a) vision-based social navigation is far from solved; (b) a critical safety deficit precedes social etiquette; (c) real-world data matters more than model scale. Crucially, we demonstrate that fine-tuning on our curated data effectively improves the success rate in real-world environments. We hope our platform provides a faithful and rigorous testbed for the next generation of vision-based social navigation research.
Related papers
- Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator [70.34667754804168]
Embodied navigation aims to build agents that interpret multimodal goals, reason in 3D space, and reach target destinations reliably in the real world.<n>We introduce Image2Sim, a real-time neural simulation framework that constructs high-quality interactive environments from posed RGB-D image sequences.
arXiv Detail & Related papers (2026-07-07T02:42:41Z) - Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting [83.60803397468139]
Habitat-GS is a navigation-centric embodied AI simulator that integrates 3D photorealistic scene rendering and drivable gaussian avatars.<n>For dynamic human modeling, we introduce a gaussian avatar module that enables each avatar to simultaneously serve as a photorealistic visual entity and an effective navigation obstacle.
arXiv Detail & Related papers (2026-04-14T11:52:59Z) - ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
arXiv Detail & Related papers (2026-03-30T15:06:41Z) - ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving Obstacles [18.27305975041577]
We propose ReaDy-Go, a novel real-to-sim simulation pipeline that synthesizes dynamic scenarios in target environments.<n>The proposed simulator generates thousands of photorealistic navigation scenarios with animatable human GS avatars from arbitrary viewpoints.<n>ReaDy-Go outperforms baselines across target environments in both simulation and real-world experiments.
arXiv Detail & Related papers (2026-02-12T04:48:18Z) - From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection [57.74400052368147]
This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning.<n>The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths.<n>Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions.
arXiv Detail & Related papers (2026-02-09T18:46:12Z) - GSON: A Group-based Social Navigation Framework with Large Multimodal Model [9.94576166903495]
This paper introduces GSON, a novel group-based social navigation framework.<n>GSON uses visual prompting to enable zero-shot extraction of social relationships among pedestrians.<n>We validate GSON through extensive real-world mobile robot navigation experiments.
arXiv Detail & Related papers (2024-09-26T17:27:15Z) - From Cognition to Precognition: A Future-Aware Framework for Social Navigation [1.9094009409000596]
We propose a reinforcement learning architecture, namely Falcon, to tackle socially-aware navigation.<n>We introduce a novel SocialNav benchmark containing two new datasets, Social-HM3D and Social-MP3D.<n>We conduct a detailed experimental analysis with the state-of-the-art learning-based method and two classic rule-based path-planning algorithms.
arXiv Detail & Related papers (2024-09-20T06:08:24Z) - Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior.<n>Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
arXiv Detail & Related papers (2023-12-26T18:56:49Z) - Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of
Demonstrations for Social Navigation [92.66286342108934]
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a'socially compliant' manner in the presence of other intelligent agents such as humans.
Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human teleoperated driving demonstrations.
arXiv Detail & Related papers (2022-03-28T19:09:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.