Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportation

• URL: http://arxiv.org/abs/2504.18010v2
• Date: Tue, 27 May 2025 14:09:11 GMT
• Title: Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportation
• Abstract summary: This paper introduces Sky-Drive, a novel distributed multi-agent simulation platform.<n>Sky-Drive supports diverse applications such as autonomous vehicle-human road users interaction modeling, human-in-the-loop training, socially-aware reinforcement learning, personalized driving development, and customized scenario generation.
• Score: 8.821410584051648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in autonomous system simulation platforms have significantly enhanced the safe and scalable testing of driving policies. However, existing simulators do not yet fully meet the needs of future transportation research-particularly in enabling effective human-AI collaboration and modeling socially-aware driving agents. This paper introduces Sky-Drive, a novel distributed multi-agent simulation platform that addresses these limitations through four key innovations: (a) a distributed architecture for synchronized simulation across multiple terminals; (b) a multi-modal human-in-the-loop framework integrating diverse sensors to collect rich behavioral data; (c) a human-AI collaboration mechanism supporting continuous and adaptive knowledge exchange; and (d) a digital twin framework for constructing high-fidelity virtual replicas of real-world transportation environments. Sky-Drive supports diverse applications such as autonomous vehicle-human road users interaction modeling, human-in-the-loop training, socially-aware reinforcement learning, personalized driving development, and customized scenario generation. Future extensions will incorporate foundation models for context-aware decision support and hardware-in-the-loop testing for real-world validation. By bridging scenario generation, data collection, algorithm training, and hardware integration, Sky-Drive has the potential to become a foundational platform for the next generation of human-centered and socially-aware autonomous transportation systems research. The demo video and code are available at:https://sky-lab-uw.github.io/Sky-Drive-website/

Related papers

• GROVE: Grounded Pedestrian Simulation via Natural Language for Interactive Social Robot Navigation [3.066376644627784]
  Pedestrian simulation is a critical component for training and deploying social robot navigation approaches.<n>GROVE is a text-to-scenario pedestrian simulation framework that combines state-of-the-art approaches to produce realistic, socially challenging scenarios.<n>The human simulation is directly integrated into Isaac Sim, Gazebo, and RViz simulators for robot deployment in highly social environments.
  arXiv  Detail & Related papers  (2026-06-24T07:39:01Z)
• On Improving Multimodal Pedestrian Trajectory Prediction with CVAE: A Study on Benchmark and Robot Data [3.857761563372033]
  Social Spatio-Temporal Graph Convolutional Neural Networks (Social-STGCNN) have shown strong performance by modeling social interactions.<n>We build on a Social-STGCNN backbone and introduce a Variational Autoencoder (CVAE)-based probabilistic formulation to explicitly model multimodal future trajectories.<n>We evaluate the method on the ETH and UCY pedestrian trajectory datasets as well as on a real-world pedestrian dataset collected by a mobile robot.
  arXiv  Detail & Related papers  (2026-05-18T11:59:52Z)
• CARLA-Air: Fly Drones Inside a CARLA World -- A Unified Infrastructure for Air-Ground Embodied Intelligence [2.061094726550731]
  We present CARLA-Air, an open-source infrastructure that unifies high-fidelity urban driving and physics-accurate multirotor flight within a single Unreal Engine process.<n>Within a shared physics tick and rendering pipeline, CARLA-Air delivers environments with rule-compliant traffic, socially-aware pedestrians, and aerodynamically consistent UAV dynamics.
  arXiv  Detail & Related papers  (2026-03-30T04:49:29Z)
• Toward a Fully Autonomous, AI-Native Particle Accelerator [0.342658286826597]
  We propose that future facilities be designed through artificial intelligence (AI) co-design.<n>Rather than retrofitting AI onto human-centric systems, we envision facilities designed from the ground up as AI-native platforms.<n>This roadmap aims to guide the accelerator community toward a future where AI-driven design and operation deliver unprecedented science output and reliability.
  arXiv  Detail & Related papers  (2026-02-19T16:49:36Z)
• aerial-autonomy-stack -- a Faster-than-real-time, Autopilot-agnostic, ROS2 Framework to Simulate and Deploy Perception-based Drones [0.3262230127283452]
  We introduce aerial-autonomy-stack, an open-source, end-to-end framework designed to streamline the pipeline from ( GPU-accelerated) perception to (flight controller-based) action.<n>We show that it supports over 20x faster-than-real-time, end-to-end simulation of a complete development and deployment stack.
  arXiv  Detail & Related papers  (2026-02-06T23:29:33Z)
• Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning [72.43357471969564]
  Isaac Lab combines high-fidelity GPU parallel physics, rendering, and a modular, composable architecture for designing environments and training robot policies.<n>We highlight its application to a diverse set of challenges, including whole-body control, cross-embodiment mobility, contact-rich and dexterous manipulation, and the integration of human demonstrations for skill acquisition.<n>We believe Isaac Lab's combination of advanced simulation capabilities, rich sensing, and data-center scale execution will help unlock the next generation of breakthroughs in robotics research.
  arXiv  Detail & Related papers  (2025-11-06T21:43:02Z)
• Confidence-Regulated Generative Diffusion Models for Reliable AI Agent Migration in Vehicular Metaverses [55.70043755630583]
  vehicular AI agents are endowed with environment perception, decision-making, and action execution capabilities.<n>We propose a reliable vehicular AI agent migration framework, achieving reliable dynamic migration and efficient resource scheduling.<n>We develop a Confidence-regulated Generative Diffusion Model (CGDM) to efficiently generate AI agent migration decisions.
  arXiv  Detail & Related papers  (2025-05-19T05:04:48Z)
• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Generative Diffusion-based Contract Design for Efficient AI Twins Migration in Vehicular Embodied AI Networks [55.15079732226397]
  Embodied AI is a rapidly advancing field that bridges the gap between cyberspace and physical space. In VEANET, embodied AI twins act as in-vehicle AI assistants to perform diverse tasks supporting autonomous driving.
  arXiv  Detail & Related papers  (2024-10-02T02:20:42Z)
• Tactics2D: A Highly Modular and Extensible Simulator for Driving Decision-making [24.795867304772404]
  Existing simulators often fall short in diverse scenarios or interactive behavior models for traffic participants. Tactics2D adopts a modular approach to traffic scenario construction, encompassing road elements, traffic regulations, behavior models, physics simulations for vehicles, and event detection mechanisms. Users can effectively evaluate the performance of driving decision-making models across various scenarios by leveraging both public datasets and user-collected real-world data.
  arXiv  Detail & Related papers  (2023-11-18T12:31:34Z)
• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [114.69732301904419]
  We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
  arXiv  Detail & Related papers  (2023-10-26T17:56:35Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses [130.15554653948897]
  In vehicular mixed reality (MR) Metaverse, distance between physical and virtual entities can be overcome. Large-scale traffic and driving simulation via realistic data collection and fusion from the physical world is difficult and costly. We propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations.
  arXiv  Detail & Related papers  (2023-02-16T16:54:10Z)
• VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles [131.2240621036954]
  We present VISTA, an open source, data-driven simulator that integrates multiple types of sensors for autonomous vehicles. Using high fidelity, real-world datasets, VISTA represents and simulates RGB cameras, 3D LiDAR, and event-based cameras. We demonstrate the ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle.
  arXiv  Detail & Related papers  (2021-11-23T18:58:10Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• Cloud2Edge Elastic AI Framework for Prototyping and Deployment of AI Inference Engines in Autonomous Vehicles [1.688204090869186]
  This paper proposes a novel framework for developing AI Inference Engines for autonomous driving applications based on deep learning modules. We introduce a simple yet elegant solution for the AI components development cycle, where prototyping takes place in the cloud according to the Software-in-the-Loop (SiL) paradigm. The effectiveness of the proposed framework is demonstrated using two real-world use-cases of AI inference engines for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-09-23T09:23:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.