Exhaustivity and anti-exhaustivity in the RSA framework: Testing the effect of prior beliefs

• URL: http://arxiv.org/abs/2202.07023v1
• Date: Mon, 14 Feb 2022 20:35:03 GMT
• Title: Exhaustivity and anti-exhaustivity in the RSA framework: Testing the effect of prior beliefs
• Authors: Alexandre Cremers and Ethan G. Wilcox and Benjamin Spector
• Abstract summary: We focus on cases when sensitivity to priors leads to counterintuitive predictions of the Rational Speech Act (RSA) framework. We show that in the baseline RSA model, under certain conditions, anti-exhaustive readings are predicted. We find no anti-exhaustivity effects, but observed that message choice is sensitive to priors, as predicted by the RSA framework overall.
• Score: 68.8204255655161
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: During communication, the interpretation of utterances is sensitive to a listener's probabilistic prior beliefs, something which is captured by one currently influential model of pragmatics, the Rational Speech Act (RSA) framework. In this paper we focus on cases when this sensitivity to priors leads to counterintuitive predictions of the framework. Our domain of interest is exhaustivity effects, whereby a sentence such as "Mary came" is understood to mean that only Mary came. We show that in the baseline RSA model, under certain conditions, anti-exhaustive readings are predicted (e.g., "Mary came" would be used to convey that both Mary and Peter came). The specific question we ask is the following: should exhaustive interpretations be derived as purely pragmatic inferences (as in the classical Gricean view, endorsed in the baseline RSA model), or should they rather be generated by an encapsulated semantic mechanism (as argued in some of the recent formal literature)? To answer this question, we provide a detailed theoretical analysis of different RSA models and evaluate them against data obtained in a new study which tested the effects of prior beliefs on both production and comprehension, improving on previous empirical work. We found no anti-exhaustivity effects, but observed that message choice is sensitive to priors, as predicted by the RSA framework overall. The best models turn out to be those which include an encapsulated exhaustivity mechanism (as other studies concluded on the basis of very different data). We conclude that, on the one hand, in the division of labor between semantics and pragmatics, semantics plays a larger role than is often thought, but, on the other hand, the tradeoff between informativity and cost which characterizes all RSA models does play a central role for genuine pragmatic effects.

Related papers

• Counterfactual Generation from Language Models [64.55296662926919]
  We show that counterfactual reasoning is conceptually distinct from interventions. We propose a framework for generating true string counterfactuals. Our experiments demonstrate that the approach produces meaningful counterfactuals.
  arXiv  Detail & Related papers  (2024-11-11T17:57:30Z)
• Towards a Fully Interpretable and More Scalable RSA Model for Metaphor Understanding [0.8437187555622164]
  The Rational Speech Act (RSA) model provides a flexible framework to model pragmatic reasoning in computational terms. Here, we introduce a new RSA framework for metaphor understanding that addresses limitations by providing an explicit formula. The model was tested against 24 metaphors, not limited to the conventional $textitJohn-is-a-shark$ type.
  arXiv  Detail & Related papers  (2024-04-03T18:09:33Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• Pragmatic Reasoning in Structured Signaling Games [2.28438857884398]
  We introduce a structured signaling game, an extension of the classical signaling game with a similarity structure between meanings in the context. We show that pragmatic agents using sRSA on top of semantic representations attain efficiency very close to the information theoretic limit. We also explore the interaction between pragmatic reasoning and learning in multi-agent reinforcement learning framework.
  arXiv  Detail & Related papers  (2023-05-17T12:43:29Z)
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests [87.60900567941428]
  A spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character. We study stress testing using the tools of causal inference.
  arXiv  Detail & Related papers  (2021-05-31T14:39:38Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• On Shapley Credit Allocation for Interpretability [1.52292571922932]
  We emphasize the importance of asking the right question when interpreting the decisions of a learning model. This paper quantifies feature relevance by weaving different natures of interpretations together with different measures as characteristic functions for Shapley symmetrization.
  arXiv  Detail & Related papers  (2020-12-10T08:25:32Z)
• Learning to refer informatively by amortizing pragmatic reasoning [35.71540493379324]
  We explore the idea that speakers might learn to amortize the cost of Rational Speech Acts over time. We find that our amortized model is able to quickly generate language that is effective and concise across a range of contexts.
  arXiv  Detail & Related papers  (2020-05-31T02:52:22Z)
• A Rate-Distortion view of human pragmatic reasoning [3.9425618017443322]
  We present a novel analysis of the Rational Speech Act (RSA) framework. We show that RSA implements an alternating for optimizing a tradeoff between expected utility and communicative effort. This work furthers the mathematical understanding of RSA models, and suggests that general information-theoretic principles may give rise to human pragmatic reasoning.
  arXiv  Detail & Related papers  (2020-05-13T22:04:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.