MetaUrban: A Simulation Platform for Embodied AI in Urban Spaces
- URL: http://arxiv.org/abs/2407.08725v1
- Date: Thu, 11 Jul 2024 17:56:49 GMT
- Title: MetaUrban: A Simulation Platform for Embodied AI in Urban Spaces
- Authors: Wayne Wu, Honglin He, Yiran Wang, Chenda Duan, Jack He, Zhizheng Liu, Quanyi Li, Bolei Zhou,
- Abstract summary: Recent advances in Robotics and Embodied AI make public urban spaces no longer exclusive to humans.
Food delivery bots and electric wheelchairs have started sharing sidewalks with pedestrians, while diverse robot dogs and humanoids have recently emerged in the street.
Ensuring the generalizability and safety of these forthcoming mobile machines is crucial when navigating through the bustling streets in urban spaces.
We present MetaUrban, a compositional simulation platform for Embodied AI research in urban spaces.
- Score: 52.0930915607703
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Public urban spaces like streetscapes and plazas serve residents and accommodate social life in all its vibrant variations. Recent advances in Robotics and Embodied AI make public urban spaces no longer exclusive to humans. Food delivery bots and electric wheelchairs have started sharing sidewalks with pedestrians, while diverse robot dogs and humanoids have recently emerged in the street. Ensuring the generalizability and safety of these forthcoming mobile machines is crucial when navigating through the bustling streets in urban spaces. In this work, we present MetaUrban, a compositional simulation platform for Embodied AI research in urban spaces. MetaUrban can construct an infinite number of interactive urban scenes from compositional elements, covering a vast array of ground plans, object placements, pedestrians, vulnerable road users, and other mobile agents' appearances and dynamics. We design point navigation and social navigation tasks as the pilot study using MetaUrban for embodied AI research and establish various baselines of Reinforcement Learning and Imitation Learning. Experiments demonstrate that the compositional nature of the simulated environments can substantially improve the generalizability and safety of the trained mobile agents. MetaUrban will be made publicly available to provide more research opportunities and foster safe and trustworthy embodied AI in urban spaces.
Related papers
- UrbanWorld: An Urban World Model for 3D City Generation [15.095017388300947]
UrbanWorld is the first generative urban world model that can automatically create a customized, realistic and interactive 3D urban world with flexible control conditions.
The crafted high-fidelity 3D urban environments enable realistic feedback and interactions for general AI and machine perceptual systems in simulations.
arXiv Detail & Related papers (2024-07-16T17:59:29Z) - GRUtopia: Dream General Robots in a City at Scale [65.08318324604116]
This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots.
GRScenes includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments.
GRResidents is a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction.
arXiv Detail & Related papers (2024-07-15T17:40:46Z) - Urban Scene Diffusion through Semantic Occupancy Map [49.20779809250597]
UrbanDiffusion is a 3D diffusion model conditioned on a Bird's-Eye View (BEV) map.
Our model learns the data distribution of scene-level structures within a latent space.
After training on real-world driving datasets, our model can generate a wide range of diverse urban scenes.
arXiv Detail & Related papers (2024-03-18T11:54:35Z) - Urban Generative Intelligence (UGI): A Foundational Platform for Agents
in Embodied City Environment [32.53845672285722]
Urban environments, characterized by their complex, multi-layered networks, face significant challenges in the face of rapid urbanization.
Recent developments in big data, artificial intelligence, urban computing, and digital twins have laid the groundwork for sophisticated city modeling and simulation.
This paper proposes Urban Generative Intelligence (UGI), a novel foundational platform integrating Large Language Models (LLMs) into urban systems.
arXiv Detail & Related papers (2023-12-19T03:12:13Z) - Learning Human-to-Robot Handovers from Point Clouds [63.18127198174958]
We propose the first framework to learn control policies for vision-based human-to-robot handovers.
We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.
arXiv Detail & Related papers (2023-03-30T17:58:36Z) - Smart Cities: Striking a Balance Between Urban Resilience and Civil
Liberties [0.0]
Cities are becoming smarter and more resilient by integrating urban infrastructure with information technology.
Concerns grow that smart cities might reverse progress on civil liberties when sensing, profiling, and predicting citizen activities.
In response, cities need to deploy technical breakthroughs, such as privacy-enhancing technologies, cohort modelling, and fair and explainable machine learning.
arXiv Detail & Related papers (2023-03-26T01:09:11Z) - Smart City Intersections: Intelligence Nodes for Future Metropolises [8.690266225071772]
Traffic intersections are the most suitable locations for the deployment of computing, communications, and intelligence services for smart cities of the future.
This paper focuses on high-bandwidth, low-latency applications, and in that context it describes: (i) system design considerations for smart city intersection intelligence nodes; (ii) key technological components including sensors, networking, edge computing, low latency design, and AI-based intelligence; and (iii) applications such as privacy preservation, cloud-connected vehicles, a real-time "radar-screen", traffic management, and monitoring of pedestrian behavior during pandemics.
arXiv Detail & Related papers (2022-05-03T17:22:57Z) - Explainable, automated urban interventions to improve pedestrian and
vehicle safety [0.8620335948752805]
This paper combines public data sources, large-scale street imagery and computer vision techniques to approach pedestrian and vehicle safety.
The steps involved in this pipeline include the adaptation and training of a Residual Convolutional Neural Network to determine a hazard index for each given urban scene.
The outcome of this computational approach is a fine-grained map of hazard levels across a city, and an identify interventions that might simultaneously improve pedestrian and vehicle safety.
arXiv Detail & Related papers (2021-10-22T09:17:39Z) - Smart Urban Mobility: When Mobility Systems Meet Smart Data [55.456196356335745]
Cities around the world are expanding dramatically, with urban population growth reaching nearly 2.5 billion people in urban areas and road traffic growth exceeding 1.2 billion cars by 2050.
The economic contribution of the transport sector represents 5% of the GDP in Europe and costs an average of US $482.05 billion in the U.S.
arXiv Detail & Related papers (2020-05-09T13:53:01Z) - Learning to Move with Affordance Maps [57.198806691838364]
The ability to autonomously explore and navigate a physical space is a fundamental requirement for virtually any mobile autonomous agent.
Traditional SLAM-based approaches for exploration and navigation largely focus on leveraging scene geometry.
We show that learned affordance maps can be used to augment traditional approaches for both exploration and navigation, providing significant improvements in performance.
arXiv Detail & Related papers (2020-01-08T04:05:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.