Modelling Lexical Ambiguity with Density Matrices

• URL: http://arxiv.org/abs/2010.05670v1
• Date: Mon, 12 Oct 2020 13:08:45 GMT
• Title: Modelling Lexical Ambiguity with Density Matrices
• Authors: Francois Meyer and Martha Lewis
• Abstract summary: We present three new neural models for learning density matrices from a corpus. Test their ability to discriminate between word senses on a range of compositional datasets.
• Score: 3.7692411550925664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Words can have multiple senses. Compositional distributional models of meaning have been argued to deal well with finer shades of meaning variation known as polysemy, but are not so well equipped to handle word senses that are etymologically unrelated, or homonymy. Moving from vectors to density matrices allows us to encode a probability distribution over different senses of a word, and can also be accommodated within a compositional distributional model of meaning. In this paper we present three new neural models for learning density matrices from a corpus, and test their ability to discriminate between word senses on a range of compositional datasets. When paired with a particular composition method, our best model outperforms existing vector-based compositional models as well as strong sentence encoders.

Related papers

• Density Matrices for Metaphor Understanding [12.568794861914451]
  We consider metaphor as a type of lexical ambiguity, and examine whether metaphorical meaning can be effectively modelled using mixtures of word senses. Our best-performing density matrix method outperforms simple baselines as well as some neural language models.
  arXiv  Detail & Related papers  (2024-08-12T11:21:56Z)
• Domain Embeddings for Generating Complex Descriptions of Concepts in Italian Language [65.268245109828]
  We propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries. The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface. Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge.
  arXiv  Detail & Related papers  (2024-02-26T15:04:35Z)
• TERM Model: Tensor Ring Mixture Model for Density Estimation [48.622060998018206]
  In this paper, we take tensor ring decomposition for density estimator, which significantly reduces the number of permutation candidates. A mixture model that incorporates multiple permutation candidates with adaptive weights is further designed, resulting in increased expressive flexibility. This approach acknowledges that suboptimal permutations can offer distinctive information besides that of optimal permutations.
  arXiv  Detail & Related papers  (2023-12-13T11:39:56Z)
• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation [52.270712965271656]
  We propose a new model of contextual word representation, not from a neural perspective, but from a purely syntactic and probabilistic perspective. We find that the graph of our model resembles transformers, with correspondences between dependencies and self-attention. Experiments show that our model performs competitively to transformers on small to medium sized datasets.
  arXiv  Detail & Related papers  (2023-11-26T06:56:02Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Kernel Density Matrices for Probabilistic Deep Learning [8.486487001779416]
  In quantum mechanics, a density matrix is the most general way to describe the state of a quantum system. This paper introduces a novel approach to probabilistic deep learning, kernel density matrices. It provides a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables.
  arXiv  Detail & Related papers  (2023-05-26T12:59:58Z)
• Linear Spaces of Meanings: Compositional Structures in Vision-Language Models [110.00434385712786]
  We investigate compositional structures in data embeddings from pre-trained vision-language models (VLMs) We first present a framework for understanding compositional structures from a geometric perspective. We then explain what these structures entail probabilistically in the case of VLM embeddings, providing intuitions for why they arise in practice.
  arXiv  Detail & Related papers  (2023-02-28T08:11:56Z)
• Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference [3.1727619150610837]
  In a bag-of-words model, the senses of a word with multiple meanings are represented as probability distributions over context words. We adapt an existing generative sense change model to develop a simpler model for the main effects of sense and time. We carry out automatic sense-annotation of snippets containing "kosmos" using our model, and measure the time-evolution of its three senses and their prevalence.
  arXiv  Detail & Related papers  (2021-04-14T11:40:21Z)
• Learning with Density Matrices and Random Features [44.98964870180375]
  A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems. This paper explores how density matrices can be used as a building block for machine learning models.
  arXiv  Detail & Related papers  (2021-02-08T17:54:59Z)
• Quantum Inspired Word Representation and Computation [13.35038288273036]
  We represent words as density matrices, which are inherently capable of representing mixed states. The experiment shows that the density matrix representation can effectively capture different aspects of word meaning. We propose a novel method to combine the coherent summation and incoherent summation in the computation of both vectors and density matrices.
  arXiv  Detail & Related papers  (2020-04-06T14:37:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.