Towards Interactive Language Modeling

• URL: http://arxiv.org/abs/2112.11911v1
• Date: Tue, 14 Dec 2021 18:35:02 GMT
• Title: Towards Interactive Language Modeling
• Authors: Maartje ter Hoeve, Evgeny Kharitonov, Dieuwke Hupkes, Emmanuel Dupoux
• Abstract summary: Motivated by these considerations, we pioneer the space of interactive language modeling. We present a road map in which we detail the steps that need to be taken towards interactive language modeling. This work aims to be the start of a larger research agenda on interactive language modeling.
• Score: 18.925337115380703
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interaction between caregivers and children plays a critical role in human language acquisition and development. Given this observation, it is remarkable that explicit interaction plays little to no role in artificial language modeling -- which also targets the acquisition of human language, yet by artificial models. Moreover, an interactive approach to language modeling has the potential to make language models substantially more versatile and to considerably impact downstream applications. Motivated by these considerations, we pioneer the space of interactive language modeling. As a first contribution we present a road map in which we detail the steps that need to be taken towards interactive language modeling. We then lead by example and take the first steps on this road map, showing the initial feasibility of our approach. As such, this work aims to be the start of a larger research agenda on interactive language modeling.

Related papers

• Modeling language contact with the Iterated Learning Model [0.0]
  Iterated learning models are agent-based models of language change. A recently introduced type of iterated learning model, the Semi-Supervised ILM is used to simulate language contact.
  arXiv  Detail & Related papers  (2024-06-11T01:43:23Z)
• LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models [50.259006481656094]
  We present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer. We present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.
  arXiv  Detail & Related papers  (2024-04-03T23:57:34Z)
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control [140.48218261864153]
  We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control. Our approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training.
  arXiv  Detail & Related papers  (2023-07-28T21:18:02Z)
• Feature Interactions Reveal Linguistic Structure in Language Models [2.0178765779788495]
  We study feature interactions in the context of feature attribution methods for post-hoc interpretability. We work out a grey box methodology, in which we train models to perfection on a formal language classification task. We show that under specific configurations, some methods are indeed able to uncover the grammatical rules acquired by a model.
  arXiv  Detail & Related papers  (2023-06-21T11:24:41Z)
• PaLM-E: An Embodied Multimodal Language Model [101.29116156731762]
  We propose embodied language models to incorporate real-world continuous sensor modalities into language models. We train these encodings end-to-end, in conjunction with a pre-trained large language model, for multiple embodied tasks. Our largest model, PaLM-E-562B with 562B parameters, is a visual-language generalist with state-of-the-art performance on OK-VQA.
  arXiv  Detail & Related papers  (2023-03-06T18:58:06Z)
• Learning Joint Representation of Human Motion and Language [22.29342443400645]
  We present MoLang (a Motion-Language connecting model) for learning joint representation of human motion and language. We propose a motion-language model with contrastive learning, empowering our model to learn better generalizable representations of the human motion domain.
  arXiv  Detail & Related papers  (2022-10-27T05:32:20Z)
• Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment [63.0407314271459]
  The proposed Cross-Align achieves the state-of-the-art (SOTA) performance on four out of five language pairs. Experiments show that the proposed Cross-Align achieves the state-of-the-art (SOTA) performance on four out of five language pairs.
  arXiv  Detail & Related papers  (2022-10-09T02:24:35Z)
• Language Models are General-Purpose Interfaces [109.45478241369655]
  We propose to use language models as a general-purpose interface to various foundation models. A collection of pretrained encoders perceive diverse modalities (such as vision, and language) We propose a semi-causal language modeling objective to jointly pretrain the interface and the modular encoders.
  arXiv  Detail & Related papers  (2022-06-13T17:34:22Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• Language Acquisition is Embodied, Interactive, Emotive: a Research Proposal [2.639737913330821]
  We review the literature on the role of embodiment and emotion in the interactive setting of spoken dialogue as necessary prerequisites for language learning for human children. We sketch a model of semantics that leverages current transformer-based models and a word-level grounded model, then explain the robot-dialogue system that will make use of our semantic model.
  arXiv  Detail & Related papers  (2021-05-10T19:40:17Z)
• Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning [16.776753238108036]
  We present a method for combining multi-agent communication and traditional data-driven approaches to natural language learning. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent self-play environment that generates task-specific rewards used to adapt or modulate the model.
  arXiv  Detail & Related papers  (2020-05-14T15:32:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.