Quantum-Like Contextuality in Large Language Models

• URL: http://arxiv.org/abs/2412.16806v1
• Date: Sat, 21 Dec 2024 23:46:55 GMT
• Title: Quantum-Like Contextuality in Large Language Models
• Authors: Kin Ian Lo, Mehrnoosh Sadrzadeh, Shane Mansfield,
• Abstract summary: This paper provides the first large scale experimental evidence for a yes answer in natural language. We construct a linguistic schema modelled over a contextual quantum scenario, instantiate it in the Simple English Wikipedia and extract probability distributions for the instances. We proved that the contextual instances came from semantically similar words, by deriving an equation between degrees of contextuality and Euclidean distances of BERT's embedding vectors.
• Score: 0.7373617024876724
• License:
• Abstract: Contextuality is a distinguishing feature of quantum mechanics and there is growing evidence that it is a necessary condition for quantum advantage. In order to make use of it, researchers have been asking whether similar phenomena arise in other domains. The answer has been yes, e.g. in behavioural sciences. However, one has to move to frameworks that take some degree of signalling into account. Two such frameworks exist: (1) a signalling-corrected sheaf theoretic model, and (2) the Contextuality-by-Default (CbD) framework. This paper provides the first large scale experimental evidence for a yes answer in natural language. We construct a linguistic schema modelled over a contextual quantum scenario, instantiate it in the Simple English Wikipedia and extract probability distributions for the instances using the large language model BERT. This led to the discovery of 77,118 sheaf-contextual and 36,938,948 CbD contextual instances. We proved that the contextual instances came from semantically similar words, by deriving an equation between degrees of contextuality and Euclidean distances of BERT's embedding vectors. A regression model further reveals that Euclidean distance is indeed the best statistical predictor of contextuality. Our linguistic schema is a variant of the co-reference resolution challenge. These results are an indication that quantum methods may be advantageous in language tasks.

Related papers

• QUITE: Quantifying Uncertainty in Natural Language Text in Bayesian Reasoning Scenarios [15.193544498311603]
  We present QUITE, a dataset of real-world Bayesian reasoning scenarios with categorical random variables and complex relationships. We conduct an extensive set of experiments, finding that logic-based models outperform out-of-the-box large language models on all reasoning types. Our results provide evidence that neuro-symbolic models are a promising direction for improving complex reasoning.
  arXiv  Detail & Related papers  (2024-10-14T12:44:59Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• C-STS: Conditional Semantic Textual Similarity [70.09137422955506]
  We propose a novel task called Conditional STS (C-STS) It measures sentences' similarity conditioned on a feature described in natural language (hereon, condition) C-STS's advantages are two-fold: it reduces the subjectivity and ambiguity of STS and enables fine-grained language model evaluation through diverse natural language conditions.
  arXiv  Detail & Related papers  (2023-05-24T12:18:50Z)
• A Model of Anaphoric Ambiguities using Sheaf Theoretic Quantum-like Contextuality and BERT [0.0]
  We construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
  arXiv  Detail & Related papers  (2022-08-11T09:31:15Z)
• The Interplay between Quantum Contextuality and Wigner Negativity [0.0]
  This thesis focuses on two nonclassical behaviours: quantum contextuality and Wigner negativity. quantum contextuality is a notion superseding nonlocality that can be exhibited by quantum systems. Wigner negativity is an unsettling non-classical feature of quantum states that originates from phase-space formulation in continuous-variable quantum optics.
  arXiv  Detail & Related papers  (2022-04-19T10:05:09Z)
• On the Quantum-like Contextuality of Ambiguous Phrases [2.6381163133447836]
  We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.
  arXiv  Detail & Related papers  (2021-07-19T13:23:42Z)
• Kochen-Specker Contextuality [0.0]
  A central result in the foundations of quantum mechanics is the Kochen-Specker theorem. It states that quantum mechanics is in conflict with classical models in which the result of a measurement does not depend on which other measurements are jointly performed. This conflict is generically called quantum contextuality.
  arXiv  Detail & Related papers  (2021-02-25T17:44:33Z)
• Unnatural Language Inference [48.45003475966808]
  We find that state-of-the-art NLI models, such as RoBERTa and BART, are invariant to, and sometimes even perform better on, examples with randomly reordered words. Our findings call into question the idea that our natural language understanding models, and the tasks used for measuring their progress, genuinely require a human-like understanding of syntax.
  arXiv  Detail & Related papers  (2020-12-30T20:40:48Z)
• Commonsense Evidence Generation and Injection in Reading Comprehension [57.31927095547153]
  We propose a Commonsense Evidence Generation and Injection framework in reading comprehension, named CEGI. The framework injects two kinds of auxiliary commonsense evidence into comprehensive reading to equip the machine with the ability of rational thinking. Experiments on the CosmosQA dataset demonstrate that the proposed CEGI model outperforms the current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-05-11T16:31:08Z)
• Information-Theoretic Probing for Linguistic Structure [74.04862204427944]
  We propose an information-theoretic operationalization of probing as estimating mutual information. We evaluate on a set of ten typologically diverse languages often underrepresented in NLP research.
  arXiv  Detail & Related papers  (2020-04-07T01:06:36Z)
• Gaussian Process States: A data-driven representation of quantum many-body physics [59.7232780552418]
  We present a novel, non-parametric form for compactly representing entangled many-body quantum states. The state is found to be highly compact, systematically improvable and efficient to sample. It is also proven to be a universal approximator' for quantum states, able to capture any entangled many-body state with increasing data set size.
  arXiv  Detail & Related papers  (2020-02-27T15:54:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.