Language-Guided World Models: A Model-Based Approach to AI Control

• URL: http://arxiv.org/abs/2402.01695v2
• Date: Fri, 5 Jul 2024 02:49:47 GMT
• Title: Language-Guided World Models: A Model-Based Approach to AI Control
• Authors: Alex Zhang, Khanh Nguyen, Jens Tuyls, Albert Lin, Karthik Narasimhan,
• Abstract summary: This paper introduces the concept of Language-Guided World Models (LWMs) LWMs are probabilistic models that can simulate environments by reading texts. We take initial steps in developing robust LWMs that can generalize to compositionally novel language descriptions.
• Score: 31.9089380929602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces the concept of Language-Guided World Models (LWMs) -- probabilistic models that can simulate environments by reading texts. Agents equipped with these models provide humans with more extensive and efficient control, allowing them to simultaneously alter agent behaviors in multiple tasks via natural verbal communication. In this work, we take initial steps in developing robust LWMs that can generalize to compositionally novel language descriptions. We design a challenging world modeling benchmark based on the game of MESSENGER (Hanjie et al., 2021), featuring evaluation settings that require varying degrees of compositional generalization. Our experiments reveal the lack of generalizability of the state-of-the-art Transformer model, as it offers marginal improvements in simulation quality over a no-text baseline. We devise a more robust model by fusing the Transformer with the EMMA attention mechanism (Hanjie et al., 2021). Our model substantially outperforms the Transformer and approaches the performance of a model with an oracle semantic parsing and grounding capability. To demonstrate the practicality of this model in improving AI safety and transparency, we simulate a scenario in which the model enables an agent to present plans to a human before execution, and to revise plans based on their language feedback.

Related papers

• Masked Generative Priors Improve World Models Sequence Modelling Capabilities [19.700020499490137]
  Masked Generative Modelling has emerged as a more efficient and superior inductive bias for modelling. GIT-STORM demonstrates substantial performance gains in RL tasks on the Atari 100k benchmark. We apply Transformer-based World Models to continuous action environments for the first time, addressing a significant gap in prior research.
  arXiv  Detail & Related papers  (2024-10-10T11:52:07Z)
• Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization [0.6629765271909505]
  This paper introduces a novel approach to model alignment through weak-to-strong generalization in the context of language models. Our results suggest that this facilitation-based approach not only enhances model performance but also provides insights into the nature of model alignment.
  arXiv  Detail & Related papers  (2024-09-11T15:16:25Z)
• LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments [70.91258869156353]
  We introduce LangSuitE, a versatile and simulation-free testbed featuring 6 representative embodied tasks in textual embodied worlds. Compared with previous LLM-based testbeds, LangSuitE offers adaptability to diverse environments without multiple simulation engines. We devise a novel chain-of-thought (CoT) schema, EmMem, which summarizes embodied states w.r.t. history information.
  arXiv  Detail & Related papers  (2024-06-24T03:36:29Z)
• Evaluating the Efficacy of AI Techniques in Textual Anonymization: A Comparative Study [5.962542204378336]
  This research focuses on text anonymisation methods, focusing on Conditional Random Fields (CRF), Long Short-Term Memory (LSTM), Embeddings from Language Models (ELMo) and Transformers architecture. Preliminary results indicate that CRF, LSTM, and ELMo individually outperform traditional methods.
  arXiv  Detail & Related papers  (2024-05-09T11:29:25Z)
• STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning [82.03481509373037]
  Recently, model-based reinforcement learning algorithms have demonstrated remarkable efficacy in visual input environments. We introduce Transformer-based wORld Model (STORM), an efficient world model architecture that combines strong modeling and generation capabilities. Storm achieves a mean human performance of $126.7%$ on the Atari $100$k benchmark, setting a new record among state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-14T16:42:02Z)
• BatGPT: A Bidirectional Autoregessive Talker from Generative Pre-trained Transformer [77.28871523946418]
  BatGPT is a large-scale language model designed and trained jointly by Wuhan University and Shanghai Jiao Tong University. It is capable of generating highly natural and fluent text in response to various types of input, including text prompts, images, and audio.
  arXiv  Detail & Related papers  (2023-07-01T15:10:01Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• PaLM-E: An Embodied Multimodal Language Model [101.29116156731762]
  We propose embodied language models to incorporate real-world continuous sensor modalities into language models. We train these encodings end-to-end, in conjunction with a pre-trained large language model, for multiple embodied tasks. Our largest model, PaLM-E-562B with 562B parameters, is a visual-language generalist with state-of-the-art performance on OK-VQA.
  arXiv  Detail & Related papers  (2023-03-06T18:58:06Z)
• N-Grammer: Augmenting Transformers with latent n-grams [35.39961549040385]
  We propose a simple yet effective modification to the Transformer architecture inspired by the literature in statistical language modeling, by augmenting the model with n-grams that are constructed from a discrete latent representation of the text sequence. We evaluate our model, the N-Grammer on language modeling on the C4 data-set as well as text classification on the SuperGLUE data-set, and find that it outperforms several strong baselines such as the Transformer and the Primer.
  arXiv  Detail & Related papers  (2022-07-13T17:18:02Z)
• Factorized Neural Transducer for Efficient Language Model Adaptation [51.81097243306204]
  We propose a novel model, factorized neural Transducer, by factorizing the blank and vocabulary prediction. It is expected that this factorization can transfer the improvement of the standalone language model to the Transducer for speech recognition. We demonstrate that the proposed factorized neural Transducer yields 15% to 20% WER improvements when out-of-domain text data is used for language model adaptation.
  arXiv  Detail & Related papers  (2021-09-27T15:04:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.