Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders

• URL: http://arxiv.org/abs/2005.02590v2
• Date: Tue, 2 Jun 2020 04:01:26 GMT
• Title: Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
• Authors: Terra Blevins and Luke Zettlemoyer
• Abstract summary: A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense.
• Score: 79.38278330678965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense. The encoders are jointly optimized in the same representation space, so that sense disambiguation can be performed by finding the nearest sense embedding for each target word embedding. Our system outperforms previous state-of-the-art models on English all-words WSD; these gains predominantly come from improved performance on rare senses, leading to a 31.1% error reduction on less frequent senses over prior work. This demonstrates that rare senses can be more effectively disambiguated by modeling their definitions.

Related papers

• Word sense extension [8.939269057094661]
  We present a paradigm of word sense extension (WSE) that enables words to spawn new senses toward novel context. We develop a framework that simulates novel word sense extension by partitioning a polysemous word type into two pseudo-tokens that mark its different senses. Our framework combines cognitive models of chaining with a learning scheme that transforms a language model embedding space to support various types of word sense extension.
  arXiv  Detail & Related papers  (2023-06-09T00:54:21Z)
• We're Afraid Language Models Aren't Modeling Ambiguity [136.8068419824318]
  Managing ambiguity is a key part of human language understanding. We characterize ambiguity in a sentence by its effect on entailment relations with another sentence. We show that a multilabel NLI model can flag political claims in the wild that are misleading due to ambiguity.
  arXiv  Detail & Related papers  (2023-04-27T17:57:58Z)
• What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories [23.623074512572593]
  We aim to explore to what extent language models are capable of discerning among senses at inference time. We leverage the relation between word senses and domains, and cast Word Sense Disambiguation (WSD) as a textual entailment problem. Our results show that this approach is indeed effective, close to supervised systems.
  arXiv  Detail & Related papers  (2023-02-07T09:55:07Z)
• A Quadratic 0-1 Programming Approach for Word Sense Disambiguation [0.0]
  Word Sense Disambiguation (WSD) is the task to determine the sense of an ambiguous word in a given context. We argue the following cause as one of the major difficulties behind finding the right patterns. In this work, we approach the interactions between senses of different target words by a Quadratic Programming model (QIP) that maximizes a WSD problem.
  arXiv  Detail & Related papers  (2022-01-13T10:46:06Z)
• Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories [47.03271152494389]
  Word Sense Disambiguation aims to automatically identify the exact meaning of one word according to its context. Existing supervised models struggle to make correct predictions on rare word senses due to limited training data. We propose a gloss alignment algorithm that can align definition sentences with the same meaning from different sense inventories to collect rich lexical knowledge.
  arXiv  Detail & Related papers  (2021-10-27T00:04:33Z)
• Large Scale Substitution-based Word Sense Induction [48.49573297876054]
  We present a word-sense induction method based on pre-trained masked language models (MLMs), which can cheaply scale to large vocabularies and large corpora. The result is a corpus which is sense-tagged according to a corpus-derived sense inventory and where each sense is associated with indicative words. Evaluation on English Wikipedia that was sense-tagged using our method shows that both the induced senses, and the per-instance sense assignment, are of high quality even compared to WSD methods, such as Babelfy.
  arXiv  Detail & Related papers  (2021-10-14T19:40:37Z)
• Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation [56.830395467247016]
  We propose a model of semantic memory for WSD in a meta-learning setting. Our model is based on hierarchical variational inference and incorporates an adaptive memory update rule via a hypernetwork. We show our model advances the state of the art in few-shot WSD, supports effective learning in extremely data scarce scenarios.
  arXiv  Detail & Related papers  (2021-06-05T20:40:01Z)
• SensPick: Sense Picking for Word Sense Disambiguation [1.1429576742016154]
  We use both context and related gloss information of a target word to model the semantic relationship between the word and the set of glosses. We propose SensPick, a type of stacked bidirectional Long Short Term Memory (LSTM) network to perform the WSD task.
  arXiv  Detail & Related papers  (2021-02-10T04:52:42Z)
• Word Sense Disambiguation for 158 Languages using Word Embeddings Only [80.79437083582643]
  Disambiguation of word senses in context is easy for humans, but a major challenge for automatic approaches. We present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings.
  arXiv  Detail & Related papers  (2020-03-14T14:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.