Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines

• URL: http://arxiv.org/abs/2604.23001v1
• Date: Fri, 24 Apr 2026 20:41:59 GMT
• Title: Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
• Abstract summary: We present a systematic, data-centric analysis of VLA research organized around three pillars: datasets, benchmarks, and data engines.<n>For datasets, we categorize real-world and synthetic corpora along embodiment diversity, modality composition, and action space formulation.<n>For benchmarks, we analyze task complexity and environment structure jointly, exposing structural gaps in compositional generalization.<n>For data engines, we examine simulation-based, video-reconstruction, and automated task-generation paradigms.
• Score: 14.989533826340486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite remarkable progress in Vision--Language--Action (VLA) models, a central bottleneck remains underexamined: the data infrastructure that underlies embodied learning. In this survey, we argue that future advances in VLA will depend less on model architecture and more on the co-design of high-fidelity data engines and structured evaluation protocols. To this end, we present a systematic, data-centric analysis of VLA research organized around three pillars: datasets, benchmarks, and data engines. For datasets, we categorize real-world and synthetic corpora along embodiment diversity, modality composition, and action space formulation, revealing a persistent fidelity-cost trade-off that fundamentally constrains large-scale collection. For benchmarks, we analyze task complexity and environment structure jointly, exposing structural gaps in compositional generalization and long-horizon reasoning evaluation that existing protocols fail to address. For data engines, we examine simulation-based, video-reconstruction, and automated task-generation paradigms, identifying their shared limitations in physical grounding and sim-to-real transfer. Synthesizing these analyses, we distill four open challenges: representation alignment, multimodal supervision, reasoning assessment, and scalable data generation. Addressing them, we argue, requires treating data infrastructure as a first-class research problem rather than a background concern.

Related papers

• Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models [50.6415287154632]
  Embodied3DBench is a robot-centric benchmark targeting low-level spatial intelligence in 3D environments.<n>The benchmark spans 12 subcategories and contains over 21k high-quality question-answer pairs.
  arXiv  Detail & Related papers  (2026-05-27T20:28:56Z)
• Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap [10.048113624715151]
  Vision-and-Language Navigation for Unmanned Aerial Vehicles (UAV-VLN) represents a pivotal challenge in embodied artificial intelligence.<n>This paper provides a comprehensive and structured survey of the field, from its formal task definition to the current state of the art.
  arXiv  Detail & Related papers  (2026-04-15T09:20:02Z)
• Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs [81.43302841109349]
  Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs)<n>We propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development.
  arXiv  Detail & Related papers  (2026-04-12T06:24:07Z)
• CoMa: Contextual Massing Generation with Vision-Language Models [7.943264761730892]
  We propose an automated framework for generating building massing based on functional requirements and site context.<n>A primary obstacle to such data-driven methods has been the lack of suitable datasets.<n>We benchmark this dataset by formulating massing generation as a conditional task for Vision-Language Models.
  arXiv  Detail & Related papers  (2026-01-13T11:44:00Z)
• From Parameters to Performance: A Data-Driven Study on LLM Structure and Development [73.67759647072519]
  Large language models (LLMs) have achieved remarkable success across various domains.<n>Despite the rapid growth in model scale and capability, systematic, data-driven research on how structural configurations affect performance remains scarce.<n>We present a large-scale dataset encompassing diverse open-source LLM structures and their performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-09-14T12:20:39Z)
• Multimodal Data Storage and Retrieval for Embodied AI: A Survey [8.079598907674903]
  Embodied AI (EAI) agents interact with the physical world, generating vast, heterogeneous multimodal data streams.<n>EAI's core requirements include physical grounding, low-latency access, and dynamic scalability.<n>Our survey is based on a comprehensive review of more than 180 related studies, providing a rigorous roadmap for designing the robust, high-performance data management frameworks.
  arXiv  Detail & Related papers  (2025-08-19T15:04:02Z)
• OpenConstruction: A Systematic Synthesis of Open Visual Datasets for Data-Centric Artificial Intelligence in Construction Monitoring [4.795391174842949]
  Construction industry increasingly relies on visual data to support Artificial Intelligence (AI) and Machine Learning (ML) applications for site monitoring.<n>Despite growing interest in visual datasets, existing resources vary widely in sizes, quality, and representativeness of real-world construction conditions.<n>This study synthesizes these findings into an open-source catalog, OpenConstruction, supporting data-driven method development.
  arXiv  Detail & Related papers  (2025-08-15T13:56:21Z)
• Vision Language Action Models in Robotic Manipulation: A Systematic Review [1.1767330101986737]
  Vision Language Action (VLA) models represent a transformative shift in robotics.<n>This review presents a comprehensive and forward-looking synthesis of the VLA paradigm.<n>We analyze 102 VLA models, 26 foundational datasets, and 12 simulation platforms.
  arXiv  Detail & Related papers  (2025-07-14T18:00:34Z)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data [89.77871049500546]
  We present a unified framework for enhancing 3D spatial reasoning in pre-trained vision-language models without modifying their architecture.<n>This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes.
  arXiv  Detail & Related papers  (2025-06-04T07:36:33Z)
• Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models [104.17057231661371]
  Time series analysis is crucial for understanding dynamics of complex systems.<n>Recent advances in foundation models have led to task-agnostic Time Series Foundation Models (TSFMs) and Large Language Model-based Time Series Models (TSLLMs)<n>Their success depends on large, diverse, and high-quality datasets, which are challenging to build due to regulatory, diversity, quality, and quantity constraints.<n>This survey provides a comprehensive review of synthetic data for TSFMs and TSLLMs, analyzing data generation strategies, their role in model pretraining, fine-tuning, and evaluation, and identifying future research directions.
  arXiv  Detail & Related papers  (2025-03-14T13:53:46Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.