Language Models: A Guide for the Perplexed

• URL: http://arxiv.org/abs/2311.17301v1
• Date: Wed, 29 Nov 2023 01:19:02 GMT
• Title: Language Models: A Guide for the Perplexed
• Authors: Sofia Serrano, Zander Brumbaugh, Noah A. Smith
• Abstract summary: This tutorial aims to help narrow the gap between those who study language models and those who are intrigued and want to learn more. We offer a scientific viewpoint that focuses on questions amenable to study through experimentation. We situate language models as they are today in the context of the research that led to their development.
• Score: 51.88841610098437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given the growing importance of AI literacy, we decided to write this tutorial to help narrow the gap between the discourse among those who study language models -- the core technology underlying ChatGPT and similar products -- and those who are intrigued and want to learn more about them. In short, we believe the perspective of researchers and educators can add some clarity to the public's understanding of the technologies beyond what's currently available, which tends to be either extremely technical or promotional material generated about products by their purveyors. Our approach teases apart the concept of a language model from products built on them, from the behaviors attributed to or desired from those products, and from claims about similarity to human cognition. As a starting point, we (1) offer a scientific viewpoint that focuses on questions amenable to study through experimentation; (2) situate language models as they are today in the context of the research that led to their development; and (3) describe the boundaries of what is known about the models at this writing.

Related papers

• Generative Artificial Intelligence: A Systematic Review and Applications [7.729155237285151]
  This paper documents the systematic review and analysis of recent advancements and techniques in Generative AI. The major impact that generative AI has made to date, has been in language generation with the development of large language models. The paper ends with a discussion of Responsible AI principles, and the necessary ethical considerations for the sustainability and growth of these generative models.
  arXiv  Detail & Related papers  (2024-05-17T18:03:59Z)
• Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models [51.43538150982291]
  We study how to learn human-interpretable concepts from data. Weaving together ideas from both fields, we show that concepts can be provably recovered from diverse data.
  arXiv  Detail & Related papers  (2024-02-14T15:23:59Z)
• Construction Grammar and Language Models [4.171555557592296]
  Recent progress in deep learning has given rise to powerful models that are primarily trained on a cloze-like task. This chapter aims to foster collaboration between researchers in the fields of natural language processing and Construction Grammar.
  arXiv  Detail & Related papers  (2023-08-25T11:37:56Z)
• Towards More Human-like AI Communication: A Review of Emergent Communication Research [0.0]
  Emergent communication (Emecom) is a field of research aiming to develop artificial agents capable of using natural language. In this review, we delineate all the common proprieties we find across the literature and how they relate to human interactions. We identify two subcategories and highlight their characteristics and open challenges.
  arXiv  Detail & Related papers  (2023-08-01T14:43:10Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models [56.93604813379634]
  Self-supervised techniques for learning speech representations have been shown to develop linguistic competence from exposure to speech without the need for human labels. We propose a language-acquisition-friendly benchmark to probe spoken language models at the lexical and syntactic levels. We highlight two exciting challenges that need to be addressed for further progress: bridging the gap between text and speech and between clean speech and in-the-wild speech.
  arXiv  Detail & Related papers  (2023-06-02T12:54:38Z)
• How to Do Things with Deep Learning Code [0.0]
  We draw attention to the means by which ordinary users might interact with, and even direct, the behavior of deep learning systems. What is at stake is the possibility of achieving an informed sociotechnical consensus about the responsible applications of large language models.
  arXiv  Detail & Related papers  (2023-04-19T03:46:12Z)
• Rethinking Explainability as a Dialogue: A Practitioner's Perspective [57.87089539718344]
  We ask doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations.
  arXiv  Detail & Related papers  (2022-02-03T22:17:21Z)
• Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques [15.906959137350247]
  This survey provides an overview of the evolution of visually grounded models of spoken language over the last 20 years. We discuss the central research questions addressed, the timeline of developments, and the datasets which enabled much of this work.
  arXiv  Detail & Related papers  (2021-04-27T14:32:22Z)
• Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey [54.34370423151014]
  This paper surveys the components of modeling approaches relaying task impacts across various generation tasks such as storytelling, summarization, translation etc. We present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them.
  arXiv  Detail & Related papers  (2020-10-14T17:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.