RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English)

• URL: http://arxiv.org/abs/2105.13266v1
• Date: Thu, 27 May 2021 16:07:13 GMT
• Title: RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English)
• Authors: Sean Trott and Benjamin Bergen
• Abstract summary: We evaluate how well contextualized embeddings accommodate the continuous, dynamic nature of word meaning. We show that cosine distance systematically underestimates how similar humans find uses of the same sense of a word to be. We propose a synthesis between psycholinguistic theories of the mental lexicon and computational models of lexical semantics.
• Score: 2.792030485253753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most words are ambiguous--i.e., they convey distinct meanings in different contexts--and even the meanings of unambiguous words are context-dependent. Both phenomena present a challenge for NLP. Recently, the advent of contextualized word embeddings has led to success on tasks involving lexical ambiguity, such as Word Sense Disambiguation. However, there are few tasks that directly evaluate how well these contextualized embeddings accommodate the more continuous, dynamic nature of word meaning--particularly in a way that matches human intuitions. We introduce RAW-C, a dataset of graded, human relatedness judgments for 112 ambiguous words in context (with 672 sentence pairs total), as well as human estimates of sense dominance. The average inter-annotator agreement (assessed using a leave-one-annotator-out method) was 0.79. We then show that a measure of cosine distance, computed using contextualized embeddings from BERT and ELMo, correlates with human judgments, but that cosine distance also systematically underestimates how similar humans find uses of the same sense of a word to be, and systematically overestimates how similar humans find uses of different-sense homonyms. Finally, we propose a synthesis between psycholinguistic theories of the mental lexicon and computational models of lexical semantics.

Related papers

• We're Afraid Language Models Aren't Modeling Ambiguity [136.8068419824318]
  Managing ambiguity is a key part of human language understanding. We characterize ambiguity in a sentence by its effect on entailment relations with another sentence. We show that a multilabel NLI model can flag political claims in the wild that are misleading due to ambiguity.
  arXiv  Detail & Related papers  (2023-04-27T17:57:58Z)
• Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
  We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings. RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
  arXiv  Detail & Related papers  (2022-12-17T05:25:17Z)
• Textual Entailment Recognition with Semantic Features from Empirical Text Representation [60.31047947815282]
  A text entails a hypothesis if and only if the true value of the hypothesis follows the text. In this paper, we propose a novel approach to identifying the textual entailment relationship between text and hypothesis. We employ an element-wise Manhattan distance vector-based feature that can identify the semantic entailment relationship between the text-hypothesis pair.
  arXiv  Detail & Related papers  (2022-10-18T10:03:51Z)
• Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT [64.40111510974957]
  We test whether meaning interferes with subject-verb number agreement in English. We generate semantically well-formed and nonsensical items. We find that BERT and humans are both sensitive to our semantic manipulation.
  arXiv  Detail & Related papers  (2022-09-21T17:57:23Z)
• Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings [11.475144702935568]
  We quantify how much the contextualized embeddings of each word sense vary across contexts in typical pre-trained models. We find that word representations are position-biased, where the first words in different contexts tend to be more similar.
  arXiv  Detail & Related papers  (2022-08-20T12:27:25Z)
• A Quadratic 0-1 Programming Approach for Word Sense Disambiguation [0.0]
  Word Sense Disambiguation (WSD) is the task to determine the sense of an ambiguous word in a given context. We argue the following cause as one of the major difficulties behind finding the right patterns. In this work, we approach the interactions between senses of different target words by a Quadratic Programming model (QIP) that maximizes a WSD problem.
  arXiv  Detail & Related papers  (2022-01-13T10:46:06Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge [0.0]
  We investigate whether recent advances in NLP, specifically contextualized word embeddings, capture human-like distinctions between English word senses. We find that participants' judgments of the relatedness between senses are correlated with distances between senses in the BERT embedding space. Our findings point towards the potential utility of continuous-space representations of sense meanings.
  arXiv  Detail & Related papers  (2020-10-25T07:56:52Z)
• Speakers Fill Lexical Semantic Gaps with Context [65.08205006886591]
  We operationalise the lexical ambiguity of a word as the entropy of meanings it can take. We find significant correlations between our estimate of ambiguity and the number of synonyms a word has in WordNet. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.
  arXiv  Detail & Related papers  (2020-10-05T17:19:10Z)
• SLAM-Inspired Simultaneous Contextualization and Interpreting for Incremental Conversation Sentences [0.0]
  We propose a method to dynamically estimate the context and interpretations of polysemous words in sequential sentences. By using the SCAIN algorithm, we can sequentially optimize the interdependence between context and word interpretation while obtaining new interpretations online.
  arXiv  Detail & Related papers  (2020-05-29T16:40:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.