NOPE: A Corpus of Naturally-Occurring Presuppositions in English
- URL: http://arxiv.org/abs/2109.06987v1
- Date: Tue, 14 Sep 2021 22:03:23 GMT
- Title: NOPE: A Corpus of Naturally-Occurring Presuppositions in English
- Authors: Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan
Lee, Zhuoye Zhao, Samuel R. Bowman, Tal Linzen
- Abstract summary: We introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus.
We investigate the context-sensitivity of 10 different types of presupposition triggers.
We evaluate machine learning models' ability to predict human inferences.
- Score: 33.69537711677911
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Understanding language requires grasping not only the overtly stated content,
but also making inferences about things that were left unsaid. These inferences
include presuppositions, a phenomenon by which a listener learns about new
information through reasoning about what a speaker takes as given.
Presuppositions require complex understanding of the lexical and syntactic
properties that trigger them as well as the broader conversational context. In
this work, we introduce the Naturally-Occurring Presuppositions in English
(NOPE) Corpus to investigate the context-sensitivity of 10 different types of
presupposition triggers and to evaluate machine learning models' ability to
predict human inferences. We find that most of the triggers we investigate
exhibit moderate variability. We further find that transformer-based models
draw correct inferences in simple cases involving presuppositions, but they
fail to capture the minority of exceptional cases in which human judgments
reveal complex interactions between context and triggers.
Related papers
- Non-verbal information in spontaneous speech -- towards a new framework
of analysis [0.5559722082623594]
This paper offers an analytical schema and a technological proof-of-concept for the categorization of prosodic signals.
We present a classification process that disentangles prosodic phenomena of three orders.
Disentangling prosodic patterns can direct a theory of communication and speech organization.
arXiv Detail & Related papers (2024-03-06T08:03:05Z) - UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
We investigate the ability to model unusual, unexpected, and unlikely situations.
Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation.
We release a new English language corpus called UNcommonsense.
arXiv Detail & Related papers (2023-11-14T19:00:55Z) - DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations.
We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
arXiv Detail & Related papers (2023-06-15T10:41:23Z) - Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
We evaluate language understanding capacities on simple inference tasks that most humans find trivial.
We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments.
The models exhibit moderate to low performance on these evaluation sets.
arXiv Detail & Related papers (2023-05-24T06:41:09Z) - Natural Language Decompositions of Implicit Content Enable Better Text
Representations [56.85319224208865]
We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.
We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.
Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
arXiv Detail & Related papers (2023-05-23T23:45:20Z) - A Linguistic Investigation of Machine Learning based Contradiction
Detection Models: An Empirical Analysis and Future Perspectives [0.34998703934432673]
We analyze two Natural Language Inference data sets with respect to their linguistic features.
The goal is to identify those syntactic and semantic properties that are particularly hard to comprehend for a machine learning model.
arXiv Detail & Related papers (2022-10-19T10:06:03Z) - Probabilistic modelling of rational communication with conditionals [0.0]
We take a probabilistic approach to pragmatic reasoning about conditionals.
We show that our model uniformly explains a number of inferences attested in the literature.
arXiv Detail & Related papers (2021-05-12T08:21:25Z) - Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
We present the first discourse dataset with multiple and subjective interpretations of English conversation.
We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
arXiv Detail & Related papers (2021-04-09T16:34:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.