Toward Universal and Interpretable World Models for Open-ended Learning Agents

• URL: http://arxiv.org/abs/2409.18676v2
• Date: Tue, 15 Oct 2024 16:23:51 GMT
• Title: Toward Universal and Interpretable World Models for Open-ended Learning Agents
• Authors: Lancelot Da Costa,
• Abstract summary: We introduce a generic, compositional and interpretable class of generative world models that supports open-ended learning agents. This is a sparse class of Bayesian networks capable of approximating a broad range of processes, which provide agents with the ability to learn world models in a manner that may be both interpretable and computationally scalable.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a generic, compositional and interpretable class of generative world models that supports open-ended learning agents. This is a sparse class of Bayesian networks capable of approximating a broad range of stochastic processes, which provide agents with the ability to learn world models in a manner that may be both interpretable and computationally scalable. This approach integrating Bayesian structure learning and intrinsically motivated (model-based) planning enables agents to actively develop and refine their world models, which may lead to developmental learning and more robust, adaptive behavior.

Related papers

• Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models [93.1043186636177]
  We explore the hypothesis that people use a combination of distributed and symbolic representations to construct bespoke mental models tailored to novel situations.<n>We propose a computational implementation of this idea -- a Model Synthesis Architecture''<n>We evaluate our MSA as a model of human judgments on a novel reasoning dataset.
  arXiv  Detail & Related papers  (2025-07-16T18:01:03Z)
• A Comprehensive Survey on Continual Learning in Generative Models [35.76314482046672]
  We present a comprehensive survey of continual learning methods for mainstream generative models.<n>We categorize these approaches into three paradigms: architecture-based, regularization-based, and replay-based.<n>We analyze continual learning setups for different generative models, including training objectives, benchmarks, and core backbones.
  arXiv  Detail & Related papers  (2025-06-16T02:27:25Z)
• General agents need world models [22.608210395958224]
  We show that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment.<n>We show that this model can be extracted from the agent's policy, and that increasing the agents performance or the complexity of the goals it can achieve requires learning increasingly accurate world models.
  arXiv  Detail & Related papers  (2025-06-02T13:01:13Z)
• World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networks [55.90051810762702]
  We present a comprehensive overview of world models, highlighting their architecture, training paradigms, and applications across prediction, generation, planning, and causal reasoning.<n>We propose Wireless Dreamer, a novel world model-based reinforcement learning framework tailored for wireless edge intelligence optimization.
  arXiv  Detail & Related papers  (2025-05-31T06:43:00Z)
• AI in a vat: Fundamental limits of efficient world modelling for agent sandboxing and interpretability [84.52205243353761]
  Recent work proposes using world models to generate controlled virtual environments in which AI agents can be tested before deployment. We investigate ways of simplifying world models that remain agnostic to the AI agent under evaluation.
  arXiv  Detail & Related papers  (2025-04-06T20:35:44Z)
• AdaWorld: Learning Adaptable World Models with Latent Actions [76.50869178593733]
  We propose AdaWorld, an innovative world model learning approach that enables efficient adaptation. Key idea is to incorporate action information during the pretraining of world models. We then develop an autoregressive world model that conditions on these latent actions.
  arXiv  Detail & Related papers  (2025-03-24T17:58:15Z)
• Multimodal Dreaming: A Global Workspace Approach to World Model-Based Reinforcement Learning [2.5749046466046903]
  In Reinforcement Learning (RL), world models aim to capture how the environment evolves in response to the agent's actions. We show that performing the dreaming process inside the latent space allows for training with fewer environment steps. We conclude that the combination of GW with World Models holds great potential for improving decision-making in RL agents.
  arXiv  Detail & Related papers  (2025-02-28T15:24:17Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer. By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity [16.15952351162363]
  We introduce a new formalism, Hidden. POMDP, designed for control with adaptive world models. We demonstrate that this approach enables learning robust behaviors across a variety of non-stationary RL benchmarks.
  arXiv  Detail & Related papers  (2024-11-02T19:09:56Z)
• Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models [106.94827590977337]
  We propose a novel world model for Multi-Agent RL (MARL) that learns decentralized local dynamics for scalability. We also introduce a Perceiver Transformer as an effective solution to enable centralized representation aggregation. Results on Starcraft Multi-Agent Challenge (SMAC) show that it outperforms strong model-free approaches and existing model-based methods in both sample efficiency and overall performance.
  arXiv  Detail & Related papers  (2024-06-22T12:40:03Z)
• Universal In-Context Approximation By Prompting Fully Recurrent Models [86.61942787684272]
  We show that RNNs, LSTMs, GRUs, Linear RNNs, and linear gated architectures can serve as universal in-context approximators. We introduce a programming language called LSRL that compiles to fully recurrent architectures.
  arXiv  Detail & Related papers  (2024-06-03T15:25:13Z)
• Generalizable Imitation Learning Through Pre-Trained Representations [19.98418419179064]
  We introduce BC-ViT, an imitation learning algorithm that leverages rich DINO pre-trained Visual Transformer (ViT) patch-level embeddings to obtain better generalization when learning through demonstrations. Our learner sees the world by clustering appearance features into semantic concepts, forming stable keypoints that generalize across a wide range of appearance variations and object types.
  arXiv  Detail & Related papers  (2023-11-15T20:15:51Z)
• Abstract Interpretation for Generalized Heuristic Search in Model-Based Planning [50.96320003643406]
  Domain-general model-based planners often derive their generality by constructing searchs through the relaxation of symbolic world models. We illustrate how abstract interpretation can serve as a unifying framework for these abstractions, extending the reach of search to richer world models. Theses can also be integrated with learning, allowing agents to jumpstart planning in novel world models via abstraction-derived information.
  arXiv  Detail & Related papers  (2022-08-05T00:22:11Z)
• Language Models are General-Purpose Interfaces [109.45478241369655]
  We propose to use language models as a general-purpose interface to various foundation models. A collection of pretrained encoders perceive diverse modalities (such as vision, and language) We propose a semi-causal language modeling objective to jointly pretrain the interface and the modular encoders.
  arXiv  Detail & Related papers  (2022-06-13T17:34:22Z)
• Procedural Generalization by Planning with Self-Supervised World Models [10.119257232716834]
  We measure the generalization ability of model-based agents in comparison to their model-free counterparts. We identify three factors of procedural generalization -- planning, self-supervised representation learning, and procedural data diversity. We find that these factors do not always provide the same benefits for the task generalization.
  arXiv  Detail & Related papers  (2021-11-02T13:32:21Z)
• A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning [104.3643447579578]
  We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state. The design allows agents to learn to plan effectively, by attending to the relevant objects, leading to better out-of-distribution generalization.
  arXiv  Detail & Related papers  (2021-06-03T19:35:19Z)
• Evolutionary Planning in Latent Space [7.863826008567604]
  Planning is a powerful approach to reinforcement learning with several desirable properties. We learn a world model that enables Evolutionary Planning in Latent Space. We show how to build a model of the world by bootstrapping it with rollouts from a random policy and iteratively refining it with rollouts from an increasingly accurate planning policy.
  arXiv  Detail & Related papers  (2020-11-23T09:21:30Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.