Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective

• URL: http://arxiv.org/abs/2404.16078v2
• Date: Fri, 26 Apr 2024 09:54:28 GMT
• Title: Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective
• Authors: Vaisakh Shaj,
• Abstract summary: Devising formalisms to develop internal world models is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the prevalent use of state space models as internal world models. The structure of models in formalisms facilitates exact probabilistic inference using belief propagation, as well as end-to-end learning via backpropagation through time. These formalisms integrate the concept of uncertainty in world states, thus improving the system's capacity to emulate the nature of the real world and quantify the confidence in its predictions.
• Score: 2.61072980439312
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Machines that can replicate human intelligence with type 2 reasoning capabilities should be able to reason at multiple levels of spatio-temporal abstractions and scales using internal world models. Devising formalisms to develop such internal world models, which accurately reflect the causal hierarchies inherent in the dynamics of the real world, is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the prevalent use of state space models (SSMs) as internal world models and propose two new probabilistic formalisms namely Hidden-Parameter SSMs and Multi-Time Scale SSMs to address these drawbacks. The structure of graphical models in both formalisms facilitates scalable exact probabilistic inference using belief propagation, as well as end-to-end learning via backpropagation through time. This approach permits the development of scalable, adaptive hierarchical world models capable of representing nonstationary dynamics across multiple temporal abstractions and scales. Moreover, these probabilistic formalisms integrate the concept of uncertainty in world states, thus improving the system's capacity to emulate the stochastic nature of the real world and quantify the confidence in its predictions. The thesis also discuss how these formalisms are in line with related neuroscience literature on Bayesian brain hypothesis and predicitive processing. Our experiments on various real and simulated robots demonstrate that our formalisms can match and in many cases exceed the performance of contemporary transformer variants in making long-range future predictions. We conclude the thesis by reflecting on the limitations of our current models and suggesting directions for future research.

Related papers

• Making Large Language Models into World Models with Precondition and Effect Knowledge [1.8561812622368763]
  We show that Large Language Models (LLMs) can be induced to perform two critical world model functions. We validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics.
  arXiv  Detail & Related papers  (2024-09-18T19:28:04Z)
• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models. We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model. We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• Multi Time Scale World Models [13.710028007050035]
  We propose a probabilistic formalism to learn multi-time scale world models. Our model uses a computationally efficient inference scheme on multiple time scales for highly accurate long-horizon predictions. Experiments show that MTS3 outperforms recent methods on several system identification benchmarks.
  arXiv  Detail & Related papers  (2023-10-27T23:18:44Z)
• Predictive World Models from Real-World Partial Observations [66.80340484148931]
  We present a framework for learning a probabilistic predictive world model for real-world road environments. While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
  arXiv  Detail & Related papers  (2023-01-12T02:07:26Z)
• Abstract Interpretation for Generalized Heuristic Search in Model-Based Planning [50.96320003643406]
  Domain-general model-based planners often derive their generality by constructing searchs through the relaxation of symbolic world models. We illustrate how abstract interpretation can serve as a unifying framework for these abstractions, extending the reach of search to richer world models. Theses can also be integrated with learning, allowing agents to jumpstart planning in novel world models via abstraction-derived information.
  arXiv  Detail & Related papers  (2022-08-05T00:22:11Z)
• DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations [119.1953397679783]
  We focus on advancing the state-of-the-art in interpreting multimodal models. Our proposed approach, DIME, enables accurate and fine-grained analysis of multimodal models.
  arXiv  Detail & Related papers  (2022-03-03T20:52:47Z)
• Quantifying Multimodality in World Models [5.593667856320704]
  We propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way.
  arXiv  Detail & Related papers  (2021-12-14T09:52:18Z)
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics [20.447000858907646]
  The capability of imagining internally with a mental model of the world is vitally important for human cognition. We propose Causal World Models (CWMs) that allow unsupervised modeling of relationships between the intervened and alternative futures. We show reductions in complexity sample for reinforcement learning tasks and improvements in counterfactual physical reasoning.
  arXiv  Detail & Related papers  (2020-12-28T13:44:36Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)
• Learning Opinion Dynamics From Social Traces [25.161493874783584]
  We propose an inference mechanism for fitting a generative, agent-like model of opinion dynamics to real-world social traces. We showcase our proposal by translating a classical agent-based model of opinion dynamics into its generative counterpart. We apply our model to real-world data from Reddit to explore the long-standing question about the impact of backfire effect.
  arXiv  Detail & Related papers  (2020-06-02T14:48:17Z)
• Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples [84.8370546614042]
  Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data. Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model. Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T11:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.