Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset

• URL: http://arxiv.org/abs/2406.09383v1
• Date: Thu, 13 Jun 2024 17:56:56 GMT
• Title: Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
• Authors: Yiming Li, Zhiheng Li, Nuo Chen, Moonjun Gong, Zonglin Lyu, Zehong Wang, Peili Jiang, Chen Feng,
• Abstract summary: We present the MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle research. MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. We conduct experiments in place recognition and neural reconstruction.
• Score: 35.52789865274631
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent interactions or repeated traversals of the same place. Such information could lead to transformative enhancements in autonomous vehicles' perception, prediction, and planning capabilities. To bridge this gap, in collaboration with the self-driving company May Mobility, we present the MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle research. More specifically, MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. Each vehicle has its own route and different vehicles may appear at nearby locations. Each vehicle is equipped with a LiDAR and surround-view RGB cameras. We curate two subsets in MARS: one facilitates collaborative driving with multiple vehicles simultaneously present at the same location, and the other enables memory retrospection through asynchronous traversals of the same location by multiple vehicles. We conduct experiments in place recognition and neural reconstruction. More importantly, MARS introduces new research opportunities and challenges such as multitraversal 3D reconstruction, multiagent perception, and unsupervised object discovery. Our data and codes can be found at https://ai4ce.github.io/MARS/.

Related papers

• USTC FLICAR: A Sensors Fusion Dataset of LiDAR-Inertial-Camera for Heavy-duty Autonomous Aerial Work Robots [13.089952067224138]
  We present the USTC FLICAR dataset, which is dedicated to the development of simultaneous localization and mapping. The proposed dataset extends the typical autonomous driving sensing suite to aerial scenes. Based on the Segment Anything Model (SAM), we produce the Semantic FLICAR dataset, which provides fine-grained semantic segmentation annotations.
  arXiv  Detail & Related papers  (2023-04-04T17:45:06Z)
• V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception [49.7212681947463]
  Vehicle-to-Vehicle (V2V) cooperative perception system has great potential to revolutionize the autonomous driving industry. We present V2V4Real, the first large-scale real-world multi-modal dataset for V2V perception. Our dataset covers a driving area of 410 km, comprising 20K LiDAR frames, 40K RGB frames, 240K annotated 3D bounding boxes for 5 classes, and HDMaps.
  arXiv  Detail & Related papers  (2023-03-14T02:49:20Z)
• RMMDet: Road-Side Multitype and Multigroup Sensor Detection System for Autonomous Driving [3.8917150802484994]
  RMMDet is a road-side multitype and multigroup sensor detection system for autonomous driving. We use a ROS-based virtual environment to simulate real-world conditions. We produce local datasets and real sand table field, and conduct various experiments.
  arXiv  Detail & Related papers  (2023-03-09T12:13:39Z)
• Multi-target multi-camera vehicle tracking using transformer-based camera link model and spatial-temporal information [29.34298951501007]
  Multi-target multi-camera tracking of vehicles, i.e. tracking vehicles across multiple cameras, is a crucial application for the development of smart city and intelligent traffic system. Main challenges of MTMCT of vehicles include the intra-class variability of the same vehicle and inter-class similarity between different vehicles. We propose a transformer-based camera link model with spatial and temporal filtering to conduct cross camera tracking.
  arXiv  Detail & Related papers  (2023-01-18T22:27:08Z)
• Unsupervised Driving Event Discovery Based on Vehicle CAN-data [62.997667081978825]
  This work presents a simultaneous clustering and segmentation approach for vehicle CAN-data that identifies common driving events in an unsupervised manner. We evaluate our approach with a dataset of real Tesla Model 3 vehicle CAN-data and a two-hour driving session that we annotated with different driving events.
  arXiv  Detail & Related papers  (2023-01-12T13:10:47Z)
• aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception [0.0]
  This dataset consists of 176 scenes with synchronized and calibrated LiDAR, camera, and radar sensors covering a 360-degree field of view. The collected data was captured in highway, urban, and suburban areas during daytime, night, and rain. We trained unimodal and multimodal baseline models for 3D object detection.
  arXiv  Detail & Related papers  (2022-11-17T10:19:59Z)
• MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning [25.191567110519866]
  We develop a new driving simulation platform called MetaDrive for the study of reinforcement learning algorithms. Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings.
  arXiv  Detail & Related papers  (2021-09-26T18:34:55Z)
• Collaborative Visual Navigation [69.20264563368762]
  We propose a large-scale 3D dataset, CollaVN, for multi-agent visual navigation (MAVN) Diverse MAVN variants are explored to make our problem more general. A memory-augmented communication framework is proposed. Each agent is equipped with a private, external memory to persistently store communication information.
  arXiv  Detail & Related papers  (2021-07-02T15:48:16Z)
• Detecting 32 Pedestrian Attributes for Autonomous Vehicles [103.87351701138554]
  In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian attributes. We introduce a Multi-Task Learning (MTL) model relying on a composite field framework, which achieves both goals in an efficient way. We show competitive detection and attribute recognition results, as well as a more stable MTL training.
  arXiv  Detail & Related papers  (2020-12-04T15:10:12Z)
• SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving [96.50297622371457]
  Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to interact with diverse road users in diverse scenarios remains largely unsolved. We develop a dedicated simulation platform called SMARTS that generates diverse and competent driving interactions.
  arXiv  Detail & Related papers  (2020-10-19T18:26:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.