NOPE: A Corpus of Naturally-Occurring Presuppositions in English

• URL: http://arxiv.org/abs/2109.06987v1
• Date: Tue, 14 Sep 2021 22:03:23 GMT
• Title: NOPE: A Corpus of Naturally-Occurring Presuppositions in English
• Authors: Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan Lee, Zhuoye Zhao, Samuel R. Bowman, Tal Linzen
• Abstract summary: We introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus. We investigate the context-sensitivity of 10 different types of presupposition triggers. We evaluate machine learning models' ability to predict human inferences.
• Score: 33.69537711677911
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presuppositions, a phenomenon by which a listener learns about new information through reasoning about what a speaker takes as given. Presuppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conversational context. In this work, we introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of presupposition triggers and to evaluate machine learning models' ability to predict human inferences. We find that most of the triggers we investigate exhibit moderate variability. We further find that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to capture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers.

Related papers

• Toward Understanding In-context vs. In-weight Learning [50.24035812301655]
  We identify simplified distributional properties that give rise to the emergence and disappearance of in-context learning. We then extend the study to a full large language model, showing how fine-tuning on various collections of natural language prompts can elicit similar in-context and in-weight learning behaviour.
  arXiv  Detail & Related papers  (2024-10-30T14:09:00Z)
• Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse [54.08750245737734]
  We propose that speakers modulate information rate based on location within a hierarchically-structured model of discourse. We find that hierarchical predictors are significant predictors of a discourse's information contour and that deeply nested hierarchical predictors are more predictive than shallow ones.
  arXiv  Detail & Related papers  (2024-10-21T14:42:37Z)
• Generating novel experimental hypotheses from language models: A case study on cross-dative generalization [15.705978435313996]
  We use LMs as simulated learners to derive novel experimental hypotheses to be tested with humans. We find LMs to replicate known patterns of children's cross-dative generalization (CDG) We propose a novel hypothesis that CDG is facilitated insofar as the features of the exposure context--in particular--are harmonically aligned.
  arXiv  Detail & Related papers  (2024-08-09T14:17:36Z)
• Non-verbal information in spontaneous speech -- towards a new framework of analysis [0.5559722082623594]
  This paper offers an analytical schema and a technological proof-of-concept for the categorization of prosodic signals. We present a classification process that disentangles prosodic phenomena of three orders. Disentangling prosodic patterns can direct a theory of communication and speech organization.
  arXiv  Detail & Related papers  (2024-03-06T08:03:05Z)
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
  We investigate the ability to model unusual, unexpected, and unlikely situations. Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation. We release a new English language corpus called UNcommonsense.
  arXiv  Detail & Related papers  (2023-11-14T19:00:55Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
  We evaluate language understanding capacities on simple inference tasks that most humans find trivial. We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments. The models exhibit moderate to low performance on these evaluation sets.
  arXiv  Detail & Related papers  (2023-05-24T06:41:09Z)
• Probabilistic modelling of rational communication with conditionals [0.0]
  We take a probabilistic approach to pragmatic reasoning about conditionals. We show that our model uniformly explains a number of inferences attested in the literature.
  arXiv  Detail & Related papers  (2021-05-12T08:21:25Z)
• Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
  We present the first discourse dataset with multiple and subjective interpretations of English conversation. We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
  arXiv  Detail & Related papers  (2021-04-09T16:34:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.