Distributional Semantics, Holism, and the Instability of Meaning

• URL: http://arxiv.org/abs/2405.12084v1
• Date: Mon, 20 May 2024 14:53:25 GMT
• Title: Distributional Semantics, Holism, and the Instability of Meaning
• Authors: Jumbly Grindrod, J. D. Porter, Nat Hansen,
• Abstract summary: A standard objection to meaning holism is the charge of instability. In this article we examine whether the instability objection poses a problem for distributional models of meaning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current language models are built on the so-called distributional semantic approach to linguistic meaning that has the distributional hypothesis at its core. The distributional hypothesis involves a holistic conception of word meaning: the meaning of a word depends upon its relations to other words in the model. A standard objection to meaning holism is the charge of instability: any change in the meaning properties of a linguistic system (a human speaker, for example) would lead to many changes or possibly a complete change in the entire system. When the systems in question are trying to communicate with each other, it has been argued that instability of this kind makes communication impossible (Fodor and Lepore 1992, 1996, 1999). In this article, we examine whether the instability objection poses a problem for distributional models of meaning. First, we distinguish between distinct forms of instability that these models could exhibit, and we argue that only one such form is relevant for understanding the relation between instability and communication: what we call differential instability. Differential instability is variation in the relative distances between points in a space, rather than variation in the absolute position of those points. We distinguish differential and absolute instability by constructing two of our own models, a toy model constructed from the text of two novels, and a more sophisticated model constructed using the Word2vec algorithm from a combination of Wikipedia and SEP articles. We demonstrate the two forms of instability by showing how these models change as the corpora they are constructed from increase in size.

Related papers

• Conjuring Semantic Similarity [59.18714889874088]
  The semantic similarity between two textual expressions measures the distance between their latent'meaning' We propose a novel approach whereby the semantic similarity among textual expressions is based not on other expressions they can be rephrased as, but rather based on the imagery they evoke. Our method contributes a novel perspective on semantic similarity that not only aligns with human-annotated scores, but also opens up new avenues for the evaluation of text-conditioned generative models.
  arXiv  Detail & Related papers  (2024-10-21T18:51:34Z)
• How often are errors in natural language reasoning due to paraphrastic variability? [29.079188032623605]
  We propose a metric for evaluating the paraphrastic consistency of natural language reasoning models. We mathematically connect this metric to the proportion of a model's variance in correctness attributable to paraphrasing. We collect ParaNLU, a dataset of 7,782 human-written and validated paraphrased reasoning problems.
  arXiv  Detail & Related papers  (2024-04-17T20:11:32Z)
• "Definition Modeling: To model definitions." Generating Definitions With Little to No Semantics [0.4061135251278187]
  We present evidence that the task may not involve as much semantics as one might expect. We show how an earlier model from the literature is both rather insensitive to semantic aspects such as explicit polysemy.
  arXiv  Detail & Related papers  (2023-06-14T11:08:38Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• A Tale of Two Laws of Semantic Change: Predicting Synonym Changes with Distributional Semantic Models [1.856334276134661]
  There are two competing, apparently opposite hypotheses in the historical linguistic literature regarding how synonymous words evolve. We take a first step toward detecting whether Law of Differentiation (LD) or Law of Parallel Change (LPC) operates for given word pairs. We then propose various computational approaches to the problem using Distributional Semantic Models and grounded in recent literature on Lexical Semantic Change detection.
  arXiv  Detail & Related papers  (2023-05-30T15:50:29Z)
• Hidden variables, free choice, context-independence, and all that [0.0]
  The paper provides a systematic account of the hidden variable models (HVMs) formulated to describe systems of random variables with mutually exclusive contexts. The equivalence and unfalsifiability imply that freedom of choice and context-independent mapping are no assumptions at all. The conjunction of these two notions, however, defines a falsifiable HVM that describes noncontextuality when applied to systems with no disturbance or to consistifications of arbitrary systems.
  arXiv  Detail & Related papers  (2023-05-25T15:06:54Z)
• Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation [37.37606905433334]
  We show that measuring uncertainty in natural language is challenging because of "semantic equivalence" We introduce semantic entropy -- an entropy which incorporates linguistic invariances created by shared meanings. Our method is unsupervised, uses only a single model, and requires no modifications to off-the-shelf language models.
  arXiv  Detail & Related papers  (2023-02-19T20:10:07Z)
• On the Strong Correlation Between Model Invariance and Generalization [54.812786542023325]
  Generalization captures a model's ability to classify unseen data. Invariance measures consistency of model predictions on transformations of the data. From a dataset-centric view, we find a certain model's accuracy and invariance linearly correlated on different test sets.
  arXiv  Detail & Related papers  (2022-07-14T17:08:25Z)
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests [87.60900567941428]
  A spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character. We study stress testing using the tools of causal inference.
  arXiv  Detail & Related papers  (2021-05-31T14:39:38Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• Lexical semantic change for Ancient Greek and Latin [61.69697586178796]
  Associating a word's correct meaning in its historical context is a central challenge in diachronic research. We build on a recent computational approach to semantic change based on a dynamic Bayesian mixture model. We provide a systematic comparison of dynamic Bayesian mixture models for semantic change with state-of-the-art embedding-based models.
  arXiv  Detail & Related papers  (2021-01-22T12:04:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.