To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing

• URL: http://arxiv.org/abs/2310.07715v1
• Date: Wed, 11 Oct 2023 17:59:36 GMT
• Title: To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing
• Authors: Sireesh Gururaja, Amanda Bertsch, Clara Na, David Gray Widder, Emma Strubell
• Abstract summary: We study factors that shape NLP as a field, including culture, incentives, and infrastructure. Our interviewees identify cyclical patterns in the field, as well as new shifts without historical parallel. We conclude by discussing shared visions, concerns, and hopes for the future of NLP.
• Score: 14.15370310437262
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: NLP is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. In this work, we seek to understand how to shape our future by better understanding our past. We study factors that shape NLP as a field, including culture, incentives, and infrastructure by conducting long-form interviews with 26 NLP researchers of varying seniority, research area, institution, and social identity. Our interviewees identify cyclical patterns in the field, as well as new shifts without historical parallel, including changes in benchmark culture and software infrastructure. We complement this discussion with quantitative analysis of citation, authorship, and language use in the ACL Anthology over time. We conclude by discussing shared visions, concerns, and hopes for the future of NLP. We hope that this study of our field's past and present can prompt informed discussion of our community's implicit norms and more deliberate action to consciously shape the future.

Related papers

• The Call for Socially Aware Language Technologies [94.6762219597438]
  We argue that many of these issues share a common core: a lack of awareness of the factors, context, and implications of the social environment in which NLP operates. We argue that substantial challenges remain for NLP to develop social awareness and that we are just at the beginning of a new era for the field.
  arXiv  Detail & Related papers  (2024-05-03T18:12:39Z)
• The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey [6.516561905186376]
  The advent of Large Language Models (LLMs) represents a notable breakthrough in Natural Language Processing (NLP) We study the inherent challenges associated with extending context length and present an organized overview of the existing strategies employed by researchers. We explore whether there is a consensus within the research community regarding evaluation standards and identify areas where further agreement is needed.
  arXiv  Detail & Related papers  (2024-01-15T18:07:21Z)
• Perspectives on the State and Future of Deep Learning -- 2023 [237.1458929375047]
  The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time. The plan is to host this survey periodically until the AI singularity paperclip-frenzy-driven doomsday, keeping an updated list of topical questions and interviewing new community members for each edition.
  arXiv  Detail & Related papers  (2023-12-07T19:58:37Z)
• Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark [55.898771405172155]
  Federated learning has emerged as a promising paradigm for privacy-preserving collaboration among different parties. We provide a systematic overview of the important and recent developments of research on federated learning.
  arXiv  Detail & Related papers  (2023-11-12T06:32:30Z)
• Ling-CL: Understanding NLP Models through Linguistic Curricula [17.44112549879293]
  We employ a characterization of linguistic complexity from psycholinguistic and language acquisition research. We develop data-driven curricula to understand the underlying linguistic knowledge that models learn to address NLP tasks.
  arXiv  Detail & Related papers  (2023-10-31T01:44:33Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• An Inclusive Notion of Text [69.36678873492373]
  We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. We introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling.
  arXiv  Detail & Related papers  (2022-11-10T14:26:43Z)
• Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions [23.389491536958772]
  Vision-and-Language Navigation (VLN) is a fundamental and interdisciplinary research topic towards this goal. VLN receives increasing attention from natural language processing, computer vision, robotics, and machine learning communities. This paper serves as a thorough reference for the VLN research community.
  arXiv  Detail & Related papers  (2022-03-22T16:58:10Z)
• Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey [54.34370423151014]
  This paper surveys the components of modeling approaches relaying task impacts across various generation tasks such as storytelling, summarization, translation etc. We present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them.
  arXiv  Detail & Related papers  (2020-10-14T17:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.