Scaling Datasets for Multi-Sensor, Multi-Agent, and Multi-Domain Learning in Autonomous Systems

• URL: http://arxiv.org/abs/2606.04444v1
• Date: Wed, 03 Jun 2026 04:46:44 GMT
• Title: Scaling Datasets for Multi-Sensor, Multi-Agent, and Multi-Domain Learning in Autonomous Systems
• Abstract summary: We present a modular dataset generation pipeline that creates terabyte-scale, ground-truth-labeled data for ground, aerial, and infrastructure-based systems.<n>We show how generated data can support application-specific training and collaborative autonomy.
• Score: 10.89128750728767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing datasets cannot support large-scale learning in multi-agent, multi-sensor, or multi-domain autonomy, where diversity and coordination are essential. We present a modular dataset generation pipeline that creates terabyte-scale, ground-truth-labeled data for ground, aerial, and infrastructure-based systems using the AVstack framework and CARLA simulator. Supporting single- and multi-agent configurations with flexible sensor suites, the pipeline enables controllable experimentation across challenging conditions. Representative perception and fusion studies show how generated data can support application-specific training and collaborative autonomy.

Related papers

• More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe [52.210953881548996]
  We show that a vision-language model can achieve competitive or state-of-the-art performance at challenging remote sensing benchmarks.<n>Our model uses a single language policy that can either answer directly in text or invoke a localization tool for segmentation and grounding.<n>Our approach achieves competitive results across a broad set of benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks.
  arXiv  Detail & Related papers  (2026-07-17T13:25:44Z)
• Multimodality as Supervision: Self-Supervised Specialization to the Test Environment via Multimodality [18.35659797985076]
  Many practical applications involve devices that are equipped with a rich set of sensors that enable multimodal sensing in their test environment.<n>This presents an opportunity to apply cross-modal learning to the multimodal data sensed by these devices to learn representations.<n>We develop Test-Space Training (TST), which performs multimodal data collection in the test environment and performs self-supervised pre-training on it.
  arXiv  Detail & Related papers  (2026-07-16T08:36:54Z)
• HeteroHub: An Applicable Data Management Framework for Heterogeneous Multi-Embodied Agent System [10.1098903025112]
  Heterogeneous Multi-Embodied Agent Systems involve coordinating multiple embodied agents with diverse capabilities to accomplish tasks in dynamic environments.<n>Existing frameworks lack a unified data management infrastructure to support the real-world deployment of such systems.<n>We present textbfHeteroHub, a data-centric framework that integrates static metadata, task-aligned training corpora, and real-time data streams.
  arXiv  Detail & Related papers  (2026-03-30T04:01:05Z)
• A Hierarchical Multi-Agent System for Autonomous Discovery in Geoscientific Data Archives [0.0]
  PANGAEA-GPT is a hierarchical multi-agent framework designed for autonomous data discovery and analysis.<n>Unlike standard Large Language Model (LLM) wrappers, our architecture implements a centralized Supervisor-Worker topology.<n>We demonstrate the system's capacity to execute complex, multi-step deterministic runtime with minimal human intervention.
  arXiv  Detail & Related papers  (2026-02-24T20:37:38Z)
• OmniGen: Unified Multimodal Sensor Generation for Autonomous Driving [58.693329943871355]
  We propose OminiGen, which generates aligned multimodal sensor data in a unified framework.<n>Our approach leverages a shared Birdu 2019s Eye View (BEV) space to unify multimodal features.<n>UAE achieves multimodal sensor decoding through volume rendering, enabling accurate and flexible reconstruction.
  arXiv  Detail & Related papers  (2025-12-16T09:18:15Z)
• LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science [69.1690891731311]
  We propose a novel multi-agent communication paradigm inspired by the blackboard architecture for traditional AI models.<n>In this framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents respond based on their capabilities.<n>We evaluate our method on three benchmarks that require explicit data discovery.
  arXiv  Detail & Related papers  (2025-09-30T22:34:23Z)
• MultiSensor-Home: A Wide-area Multi-modal Multi-view Dataset for Action Recognition and Transformer-based Sensor Fusion [2.7745600113170994]
  We introduce the MultiSensor-Home dataset, a novel benchmark for comprehensive action recognition in home environments.<n>We also propose the Multi-modal Multi-view Transformer-based Sensor Fusion (MultiTSF) method.
  arXiv  Detail & Related papers  (2025-04-03T05:23:08Z)
• Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection [64.08296187555095]
  Uni$2$Det is a framework for unified and universal multi-dataset training on 3D detection. We introduce multi-stage prompting modules for multi-dataset 3D detection. Results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.
  arXiv  Detail & Related papers  (2024-09-30T17:57:50Z)
• Very Large-Scale Multi-Agent Simulation in AgentScope [112.98986800070581]
  We develop new features and components for AgentScope, a user-friendly multi-agent platform. We propose an actor-based distributed mechanism towards great scalability and high efficiency. We also provide a web-based interface for conveniently monitoring and managing a large number of agents.
  arXiv  Detail & Related papers  (2024-07-25T05:50:46Z)
• Meta-models for transfer learning in source localisation [3.8922067105369154]
  This work looks to capture the interdependencies between acoustic emission (AE) experiments (as meta-models) We utilise a Bayesian multilevel approach where a higher level meta-model captures the inter-task relationships. Key contribution is how knowledge of the experimental campaign can be encoded between tasks as well as within tasks.
  arXiv  Detail & Related papers  (2023-05-15T14:02:35Z)
• SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices [69.1412199244903]
  We present a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. S SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.
  arXiv  Detail & Related papers  (2021-09-08T22:06:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.