Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions

• URL: http://arxiv.org/abs/2604.15395v1
• Date: Thu, 16 Apr 2026 09:32:36 GMT
• Title: Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions
• Abstract summary: Foundations Models (FMs) provide capabilities in multi-modal understanding and reasoning, long-horizon planning, and cross-embodiment generalization.<n>This study provides a holistic, systematic, and in-depth review of the research landscape of FMs in robotics.
• Score: 24.63914332126002
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the recent years, the field of robotics has been undergoing a transformative paradigm shift from fixed, single-task, domain-specific solutions towards adaptive, multi-function, general-purpose agents, capable of operating in complex, open-world, and dynamic environments. This tremendous advancement is primarily driven by the emergence of Foundation Models (FMs), i.e., large-scale neural-network architectures trained on massive, heterogeneous datasets that provide unprecedented capabilities in multi-modal understanding and reasoning, long-horizon planning, and cross-embodiment generalization. In this context, the current study provides a holistic, systematic, and in-depth review of the research landscape of FMs in robotics. In particular, the evolution of the field is initially delineated through five distinct research phases, spanning from the early incorporation of Natural Language Processing (NLP) and Computer Vision (CV) models to the current frontier of multi-sensory generalization and real-world deployment. Subsequently, a highly-granular taxonomic investigation of the literature is performed, examining the following key aspects: a) the employed FM types, including LLMs, VFMs, VLMs, and VLAs, b) the underlying neural-network architectures, c) the adopted learning paradigms, d) the different learning stages of knowledge incorporation, e) the major robotic tasks, and f) the main real-world application domains. For each aspect, comparative analysis and critical insights are provided. Moreover, a report on the publicly available datasets used for model training and evaluation across the considered robotic tasks is included. Furthermore, a hierarchical discussion on the current open challenges and promising future research directions in the field is incorporated.

Related papers

• Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap [10.048113624715151]
  Vision-and-Language Navigation for Unmanned Aerial Vehicles (UAV-VLN) represents a pivotal challenge in embodied artificial intelligence.<n>This paper provides a comprehensive and structured survey of the field, from its formal task definition to the current state of the art.
  arXiv  Detail & Related papers  (2026-04-15T09:20:02Z)
• Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives [101.22313265304979]
  Recent advances in vision, language, and multimodal learning have accelerated progress in robotic foundation models.<n>This survey examines robot manipulation from an algorithmic perspective.<n>We organize recent learning-based approaches within a unified abstraction of high-level planning and low-level control.
  arXiv  Detail & Related papers  (2025-12-28T16:05:38Z)
• Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges [54.669838624278924]
  Foundation models have transformed natural language processing and computer vision.<n>With powerful generalization and transfer learning capabilities, they align naturally with the multimodal, multi-resolution, and multi-temporal characteristics of remote sensing data.<n>This survey delivers a comprehensive review of multimodal GFMs from a modality-driven perspective.
  arXiv  Detail & Related papers  (2025-10-27T03:40:00Z)
• A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools [15.928285656168422]
  Foundation models (FMs) are enabling scalable, general-purpose, and multimodal AI systems for scientific discovery.<n>This survey provides a comprehensive overview of foundation models, agentic systems, datasets, and computational tools supporting this growing field.
  arXiv  Detail & Related papers  (2025-06-25T18:10:30Z)
• Anomaly Detection and Generation with Diffusion Models: A Survey [51.61574868316922]
  Anomaly detection (AD) plays a pivotal role across diverse domains, including cybersecurity, finance, healthcare, and industrial manufacturing.<n>Recent advancements in deep learning, specifically diffusion models (DMs), have sparked significant interest.<n>This survey aims to guide researchers and practitioners in leveraging DMs for innovative AD solutions across diverse applications.
  arXiv  Detail & Related papers  (2025-06-11T03:29:18Z)
• Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision [49.073964142139495]
  We systematically review the applications and advancements of multimodal fusion methods and vision-language models.<n>For semantic scene understanding tasks, we categorize fusion approaches into encoder-decoder frameworks, attention-based architectures, and graph neural networks.<n>We identify key challenges in current research, including cross-modal alignment, efficient fusion, real-time deployment, and domain adaptation.
  arXiv  Detail & Related papers  (2025-04-03T10:53:07Z)
• Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directions [16.78870612041548]
  Embodied multimodal large models (EMLMs) have gained significant attention in recent years due to their potential to bridge the gap between perception, cognition, and action in complex, real-world environments.<n>This comprehensive review explores the development of such models, including Large Language Models (LLMs), Large Vision Models (LVMs), and other models.
  arXiv  Detail & Related papers  (2025-02-21T09:41:27Z)
• A Survey of Model Architectures in Information Retrieval [59.61734783818073]
  The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP)<n>We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs)<n>We conclude with a forward-looking discussion of emerging challenges and future directions.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• Foundation Models for Remote Sensing and Earth Observation: A Survey [101.77425018347557]
  This survey systematically reviews the emerging field of Remote Sensing Foundation Models (RSFMs)<n>It begins with an outline of their motivation and background, followed by an introduction of their foundational concepts.<n>We benchmark these models against publicly available datasets, discuss existing challenges, and propose future research directions.
  arXiv  Detail & Related papers  (2024-10-22T01:08:21Z)
• A Survey on Robotics with Foundation Models: toward Embodied AI [30.999414445286757]
  Recent advances in computer vision, natural language processing, and multi-modality learning have shown that the foundation models have superhuman capabilities for specific tasks. This survey aims to provide a comprehensive and up-to-date overview of foundation models in robotics, focusing on autonomous manipulation and encompassing high-level planning and low-level control.
  arXiv  Detail & Related papers  (2024-02-04T07:55:01Z)
• Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis [82.59451639072073]
  General-purpose robots operate seamlessly in any environment, with any object, and utilize various skills to complete diverse tasks. As a community, we have been constraining most robotic systems by designing them for specific tasks, training them on specific datasets, and deploying them within specific environments. Motivated by the impressive open-set performance and content generation capabilities of web-scale, large-capacity pre-trained models, we devote this survey to exploring how foundation models can be applied to general-purpose robotics.
  arXiv  Detail & Related papers  (2023-12-14T10:02:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.