Understanding World or Predicting Future? A Comprehensive Survey of World Models

• URL: http://arxiv.org/abs/2411.14499v1
• Date: Thu, 21 Nov 2024 03:58:50 GMT
• Title: Understanding World or Predicting Future? A Comprehensive Survey of World Models
• Authors: Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, Fengli Xu, Yong Li,
• Abstract summary: This survey offers a comprehensive review of the literature on world models. World models are regarded as tools for either understanding the present state of the world or predicting its future dynamics. We explore the application of world models in key domains, including autonomous driving, robotics, and social simulacra.
• Score: 21.96900555014452
• License:
• Abstract: The concept of world models has garnered significant attention due to advancements in multimodal large language models such as GPT-4 and video generation models such as Sora, which are central to the pursuit of artificial general intelligence. This survey offers a comprehensive review of the literature on world models. Generally, world models are regarded as tools for either understanding the present state of the world or predicting its future dynamics. This review presents a systematic categorization of world models, emphasizing two primary functions: (1) constructing internal representations to understand the mechanisms of the world, and (2) predicting future states to simulate and guide decision-making. Initially, we examine the current progress in these two categories. We then explore the application of world models in key domains, including autonomous driving, robotics, and social simulacra, with a focus on how each domain utilizes these aspects. Finally, we outline key challenges and provide insights into potential future research directions.

Related papers

• Human-Centric Foundation Models: Perception, Generation and Agentic Modeling [79.97999901785772]
  Human-centric Foundation Models unify diverse human-centric tasks into a single framework. We present a comprehensive overview of HcFMs by proposing a taxonomy that categorizes current approaches into four groups. This survey aims to serve as a roadmap for researchers and practitioners working towards more robust, versatile, and intelligent digital human and embodiments modeling.
  arXiv  Detail & Related papers  (2025-02-12T16:38:40Z)
• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling. This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
  World models and video generation are pivotal technologies in the domain of autonomous driving. This paper investigates the relationship between these two technologies. By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
  arXiv  Detail & Related papers  (2024-11-05T08:58:35Z)
• Making Large Language Models into World Models with Precondition and Effect Knowledge [1.8561812622368763]
  We show that Large Language Models (LLMs) can be induced to perform two critical world model functions. We validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics.
  arXiv  Detail & Related papers  (2024-09-18T19:28:04Z)
• Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond [101.15395503285804]
  General world models represent a crucial pathway toward achieving Artificial General Intelligence (AGI) In this survey, we embark on a comprehensive exploration of the latest advancements in world models. We examine challenges and limitations of world models, and discuss their potential future directions.
  arXiv  Detail & Related papers  (2024-05-06T14:37:07Z)
• Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective [2.61072980439312]
  Devising formalisms to develop internal world models is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the prevalent use of state space models as internal world models. The structure of models in formalisms facilitates exact probabilistic inference using belief propagation, as well as end-to-end learning via backpropagation through time. These formalisms integrate the concept of uncertainty in world states, thus improving the system's capacity to emulate the nature of the real world and quantify the confidence in its predictions.
  arXiv  Detail & Related papers  (2024-04-24T12:41:04Z)
• Open-world Machine Learning: A Review and New Outlooks [83.6401132743407]
  This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm. It aims to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.
  arXiv  Detail & Related papers  (2024-03-04T06:25:26Z)
• Predictive World Models from Real-World Partial Observations [66.80340484148931]
  We present a framework for learning a probabilistic predictive world model for real-world road environments. While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
  arXiv  Detail & Related papers  (2023-01-12T02:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.