Multi Time Scale World Models

• URL: http://arxiv.org/abs/2310.18534v3
• Date: Mon, 4 Dec 2023 10:20:40 GMT
• Title: Multi Time Scale World Models
• Authors: Vaisakh Shaj, Saleh Gholam Zadeh, Ozan Demir, Luiz Ricardo Douat, Gerhard Neumann
• Abstract summary: We propose a probabilistic formalism to learn multi-time scale world models. Our model uses a computationally efficient inference scheme on multiple time scales for highly accurate long-horizon predictions. Experiments show that MTS3 outperforms recent methods on several system identification benchmarks.
• Score: 13.710028007050035
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intelligent agents use internal world models to reason and make predictions about different courses of their actions at many scales. Devising learning paradigms and architectures that allow machines to learn world models that operate at multiple levels of temporal abstractions while dealing with complex uncertainty predictions is a major technical hurdle. In this work, we propose a probabilistic formalism to learn multi-time scale world models which we call the Multi Time Scale State Space (MTS3) model. Our model uses a computationally efficient inference scheme on multiple time scales for highly accurate long-horizon predictions and uncertainty estimates over several seconds into the future. Our experiments, which focus on action conditional long horizon future predictions, show that MTS3 outperforms recent methods on several system identification benchmarks including complex simulated and real-world dynamical systems. Code is available at this repository: https://github.com/ALRhub/MTS3.

Related papers

• A Practitioner's Guide to Continual Multimodal Pretraining [83.63894495064855]
  Multimodal foundation models serve numerous applications at the intersection of vision and language. To keep models updated, research into continual pretraining mainly explores scenarios with either infrequent, indiscriminate updates on large-scale new data, or frequent, sample-level updates. We introduce FoMo-in-Flux, a continual multimodal pretraining benchmark with realistic compute constraints and practical deployment requirements.
  arXiv  Detail & Related papers  (2024-08-26T17:59:01Z)
• Deciphering Movement: Unified Trajectory Generation Model for Multi-Agent [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs. Specifically, we introduce a Ghost Spatial Masking (GSM) module embedded within a Transformer encoder for spatial feature extraction. We benchmark three practical sports game datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective [2.61072980439312]
  Devising formalisms to develop internal world models is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the prevalent use of state space models as internal world models. The structure of models in formalisms facilitates exact probabilistic inference using belief propagation, as well as end-to-end learning via backpropagation through time. These formalisms integrate the concept of uncertainty in world states, thus improving the system's capacity to emulate the nature of the real world and quantify the confidence in its predictions.
  arXiv  Detail & Related papers  (2024-04-24T12:41:04Z)
• Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners [85.03486419424647]
  KnowNo is a framework for measuring and aligning the uncertainty of large language models. KnowNo builds on the theory of conformal prediction to provide statistical guarantees on task completion.
  arXiv  Detail & Related papers  (2023-07-04T21:25:12Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)
• Predictive World Models from Real-World Partial Observations [66.80340484148931]
  We present a framework for learning a probabilistic predictive world model for real-world road environments. While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
  arXiv  Detail & Related papers  (2023-01-12T02:07:26Z)
• Learning Behavior Representations Through Multi-Timescale Bootstrapping [8.543808476554695]
  We introduce Bootstrap Across Multiple Scales (BAMS), a multi-scale representation learning model for behavior. We first apply our method on a dataset of quadrupeds navigating in different terrain types, and show that our model captures the temporal complexity of behavior.
  arXiv  Detail & Related papers  (2022-06-14T17:57:55Z)
• TSEM: Temporally Weighted Spatiotemporal Explainable Neural Network for Multivariate Time Series [0.0]
  We present a model-agnostic, model-specific approach to time series deep learning. We show that TSEM outperforms XCM in terms of accuracy, while also satisfying a number of interpretability criteria.
  arXiv  Detail & Related papers  (2022-05-25T18:54:25Z)
• Quantifying Multimodality in World Models [5.593667856320704]
  We propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way.
  arXiv  Detail & Related papers  (2021-12-14T09:52:18Z)
• Harnessing expressive capacity of Machine Learning modeling to represent complex coupling of Earth's auroral space weather regimes [0.0]
  We develop multiple Deep Learning (DL) models that advance predictions of the global auroral particle precipitation. We use observations from low Earth orbiting spacecraft of electron energy flux to develop a model that improves global nowcasts. Notably, the ML models improve prediction of the extreme events, historically to accurate specification and indicate that increased capacity provided by ML innovation can address grand challenges in science of space weather.
  arXiv  Detail & Related papers  (2021-11-29T22:35:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.