NOPE: A Corpus of Naturally-Occurring Presuppositions in English

• URL: http://arxiv.org/abs/2109.06987v1
• Date: Tue, 14 Sep 2021 22:03:23 GMT
• Title: NOPE: A Corpus of Naturally-Occurring Presuppositions in English
• Abstract summary: We introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus. We investigate the context-sensitivity of 10 different types of presupposition triggers. We evaluate machine learning models' ability to predict human inferences.
• Score: 33.69537711677911
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presuppositions, a phenomenon by which a listener learns about new information through reasoning about what a speaker takes as given. Presuppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conversational context. In this work, we introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of presupposition triggers and to evaluate machine learning models' ability to predict human inferences. We find that most of the triggers we investigate exhibit moderate variability. We further find that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to capture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers.

Related papers

• DRInQ: Evaluating Conversational Implicature with Controlled Context Variation [24.87053225680957]
  We introduce DRinQ, a benchmark for evaluating pragmatic reasoning about conversational implicature in question utterances.<n>We propose a semi-automated pipeline that produces question-context-interpretation instances with systematic variation.
  arXiv  Detail & Related papers  (2026-05-22T22:39:32Z)
• Readers make targeted regressions to plausible errors in reanalysis of "noisy-channel garden-path" sentences [10.293916944634804]
  We study reading dynamics for noisy-channel garden-path'' sentences.<n>We find evidence for targeted regressions.<n>We discuss the implications for theories of noisy-channel language comprehension.
  arXiv  Detail & Related papers  (2026-05-18T15:42:22Z)
• The production of meaning in the processing of natural language [0.0]
  We study the Bell inequality in experiments of contextuality during interpretation of ambiguous expressions.<n>We find that the interquartile range of the $|S|$ distribution is completely orthogonal to all external benchmarks.<n>We discuss the information-theoretic constraints that genuine contextuality imposes on prompt injection defenses.
  arXiv  Detail & Related papers  (2026-03-20T18:00:19Z)
• Predicting States of Understanding in Explanatory Interactions Using Cognitive Load-Related Linguistic Cues [4.351015250928505]
  We investigate how verbal and nonverbal linguistic features, exhibited by speakers and listeners in dialogue, can contribute to predicting the listener's state of understanding.<n>Specifically, we examine three linguistic cues related to cognitive load and hypothesised to correlate with listener understanding.
  arXiv  Detail & Related papers  (2026-03-20T15:58:54Z)
• When Contextual Inference Fails: Cancelability in Interactive Instruction Following [51.2195840589474]
  We introduce Build What I Mean, an interactive benchmark for contextual meaning construction.<n>In BWIM, models must resolve ambiguity by either performing a contextual inference or requesting clarification at a small communication cost.<n>We observe suboptimal strategies, such as partner-blind over-clarification and question-averse guessing under uncertainty.
  arXiv  Detail & Related papers  (2026-03-20T14:46:59Z)
• Toward Understanding In-context vs. In-weight Learning [50.24035812301655]
  We identify simplified distributional properties that give rise to the emergence and disappearance of in-context learning. We then extend the study to a full large language model, showing how fine-tuning on various collections of natural language prompts can elicit similar in-context and in-weight learning behaviour.
  arXiv  Detail & Related papers  (2024-10-30T14:09:00Z)
• Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse [54.08750245737734]
  We propose that speakers modulate information rate based on location within a hierarchically-structured model of discourse. We find that hierarchical predictors are significant predictors of a discourse's information contour and that deeply nested hierarchical predictors are more predictive than shallow ones.
  arXiv  Detail & Related papers  (2024-10-21T14:42:37Z)
• Generating novel experimental hypotheses from language models: A case study on cross-dative generalization [15.705978435313996]
  We use LMs as simulated learners to derive novel experimental hypotheses to be tested with humans. We find LMs to replicate known patterns of children's cross-dative generalization (CDG) We propose a novel hypothesis that CDG is facilitated insofar as the features of the exposure context--in particular--are harmonically aligned.
  arXiv  Detail & Related papers  (2024-08-09T14:17:36Z)
• Non-verbal information in spontaneous speech -- towards a new framework of analysis [0.5559722082623594]
  This paper offers an analytical schema and a technological proof-of-concept for the categorization of prosodic signals. We present a classification process that disentangles prosodic phenomena of three orders. Disentangling prosodic patterns can direct a theory of communication and speech organization.
  arXiv  Detail & Related papers  (2024-03-06T08:03:05Z)
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
  We investigate the ability to model unusual, unexpected, and unlikely situations. Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation. We release a new English language corpus called UNcommonsense.
  arXiv  Detail & Related papers  (2023-11-14T19:00:55Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
  We evaluate language understanding capacities on simple inference tasks that most humans find trivial. We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments. The models exhibit moderate to low performance on these evaluation sets.
  arXiv  Detail & Related papers  (2023-05-24T06:41:09Z)
• Probabilistic modelling of rational communication with conditionals [0.0]
  We take a probabilistic approach to pragmatic reasoning about conditionals. We show that our model uniformly explains a number of inferences attested in the literature.
  arXiv  Detail & Related papers  (2021-05-12T08:21:25Z)
• Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
  We present the first discourse dataset with multiple and subjective interpretations of English conversation. We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
  arXiv  Detail & Related papers  (2021-04-09T16:34:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.