Theoretical foundations and limits of word embeddings: what types of meaning can they capture?

• URL: http://arxiv.org/abs/2107.10413v1
• Date: Thu, 22 Jul 2021 00:40:33 GMT
• Title: Theoretical foundations and limits of word embeddings: what types of meaning can they capture?
• Authors: Alina Arseniev-Koehler
• Abstract summary: Measuring meaning is a central problem in cultural sociology. I theorize the ways in which word embeddings model three core premises of a structural linguistic theory of meaning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Measuring meaning is a central problem in cultural sociology and word embeddings may offer powerful new tools to do so. But like any tool, they build on and exert theoretical assumptions. In this paper I theorize the ways in which word embeddings model three core premises of a structural linguistic theory of meaning: that meaning is relational, coherent, and may be analyzed as a static system. In certain ways, word embedding methods are vulnerable to the same, enduring critiques of these premises. In other ways, they offer novel solutions to these critiques. More broadly, formalizing the study of meaning with word embeddings offers theoretical opportunities to clarify core concepts and debates in cultural sociology, such as the coherence of meaning. Just as network analysis specified the once vague notion of social relations (Borgatti et al. 2009), formalizing meaning with embedding methods can push us to specify and reimagine meaning itself.

Related papers

• A Complexity-Based Theory of Compositionality [53.025566128892066]
  In AI, compositional representations can enable a powerful form of out-of-distribution generalization. Here, we propose a formal definition of compositionality that accounts for and extends our intuitions about compositionality. The definition is conceptually simple, quantitative, grounded in algorithmic information theory, and applicable to any representation.
  arXiv  Detail & Related papers  (2024-10-18T18:37:27Z)
• A Category-theoretical Meta-analysis of Definitions of Disentanglement [97.34033555407403]
  Disentangling the factors of variation in data is a fundamental concept in machine learning. This paper presents a meta-analysis of existing definitions of disentanglement.
  arXiv  Detail & Related papers  (2023-05-11T15:24:20Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• Towards a Theoretical Understanding of Word and Relation Representation [8.020742121274418]
  Representing words by vectors, or embeddings, enables computational reasoning. We focus on word embeddings learned from text corpora and knowledge graphs.
  arXiv  Detail & Related papers  (2022-02-01T15:34:58Z)
• On the Faithfulness Measurements for Model Interpretations [100.2730234575114]
  Post-hoc interpretations aim to uncover how natural language processing (NLP) models make predictions. To tackle these issues, we start with three criteria: the removal-based criterion, the sensitivity of interpretations, and the stability of interpretations. Motivated by the desideratum of these faithfulness notions, we introduce a new class of interpretation methods that adopt techniques from the adversarial domain.
  arXiv  Detail & Related papers  (2021-04-18T09:19:44Z)
• How to marry a star: probabilistic constraints for meaning in context [11.670687428360688]
  We derive a notion of 'word meaning in context' that characterizes meaning as both intensional and conceptual. We introduce a framework for specifying local as well as global constraints on word meaning in context, together with their interactions. We represent sentence meaning as a'situation description system', a probabilistic model which takes utterance understanding to be the mental process of describing to oneself one or more situations that would account for an observed utterance.
  arXiv  Detail & Related papers  (2020-09-16T21:11:23Z)
• Computational linguistic assessment of textbook and online learning media by means of threshold concepts in business education [59.003956312175795]
  From a linguistic perspective, threshold concepts are instances of specialized vocabularies, exhibiting particular linguistic features. The profiles of 63 threshold concepts from business education have been investigated in textbooks, newspapers, and Wikipedia. The three kinds of resources can indeed be distinguished in terms of their threshold concepts' profiles.
  arXiv  Detail & Related papers  (2020-08-05T12:56:16Z)
• Cultural Cartography with Word Embeddings [0.0]
  We show how word embeddings are commensurate with prevailing theories of meaning in sociology. First, one can hold terms constant and measure how the embedding space moves around them. Second, one can also hold the embedding space constant and see how documents or authors move relative to it.
  arXiv  Detail & Related papers  (2020-07-09T01:58:28Z)
• (Re)construing Meaning in NLP [15.37817898307963]
  We show that the way something is expressed reflects different ways of conceptualizing or construing the information being conveyed. We show how insights from construal could inform theoretical and practical work in NLP.
  arXiv  Detail & Related papers  (2020-05-18T21:21:34Z)
• A computational model implementing subjectivity with the 'Room Theory'. The case of detecting Emotion from Text [68.8204255655161]
  This work introduces a new method to consider subjectivity and general context dependency in text analysis. By using similarity measure between words, we are able to extract the relative relevance of the elements in the benchmark. This method could be applied to all the cases where evaluating subjectivity is relevant to understand the relative value or meaning of a text.
  arXiv  Detail & Related papers  (2020-05-12T21:26:04Z)
• Expressiveness and machine processability of Knowledge Organization Systems (KOS): An analysis of concepts and relations [0.0]
  The potential of both the expressiveness and machine processability of each Knowledge Organization System is extensively regulated by its structural rules. Ontologies explicitly define diverse types of relations, and are by their nature machine-processable.
  arXiv  Detail & Related papers  (2020-03-11T12:35:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.