Word sense extension

• URL: http://arxiv.org/abs/2306.05609v1
• Date: Fri, 9 Jun 2023 00:54:21 GMT
• Title: Word sense extension
• Authors: Lei Yu, Yang Xu
• Abstract summary: We present a paradigm of word sense extension (WSE) that enables words to spawn new senses toward novel context. We develop a framework that simulates novel word sense extension by partitioning a polysemous word type into two pseudo-tokens that mark its different senses. Our framework combines cognitive models of chaining with a learning scheme that transforms a language model embedding space to support various types of word sense extension.
• Score: 8.939269057094661
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans often make creative use of words to express novel senses. A long-standing effort in natural language processing has been focusing on word sense disambiguation (WSD), but little has been explored about how the sense inventory of a word may be extended toward novel meanings. We present a paradigm of word sense extension (WSE) that enables words to spawn new senses toward novel context. We develop a framework that simulates novel word sense extension by first partitioning a polysemous word type into two pseudo-tokens that mark its different senses, and then inferring whether the meaning of a pseudo-token can be extended to convey the sense denoted by the token partitioned from the same word type. Our framework combines cognitive models of chaining with a learning scheme that transforms a language model embedding space to support various types of word sense extension. We evaluate our framework against several competitive baselines and show that it is superior in predicting plausible novel senses for over 7,500 English words. Furthermore, we show that our WSE framework improves performance over a range of transformer-based WSD models in predicting rare word senses with few or zero mentions in the training data.

Related papers

• Can Word Sense Distribution Detect Semantic Changes of Words? [35.17635565325166]
  We show that word sense distributions can be accurately used to predict semantic changes of words in English, German, Swedish and Latin. Our experimental results on SemEval 2020 Task 1 dataset show that word sense distributions can be accurately used to predict semantic changes of words.
  arXiv  Detail & Related papers  (2023-10-16T13:41:27Z)
• Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models [67.19567060894563]
  Pretrained Language Models (PLMs) learn rich cross-lingual knowledge and can be finetuned to perform well on diverse tasks. We present a new study investigating how well PLMs capture cross-lingual word sense with Contextual Word-Level Translation (C-WLT) We find that as the model size increases, PLMs encode more cross-lingual word sense knowledge and better use context to improve WLT performance.
  arXiv  Detail & Related papers  (2023-04-26T19:55:52Z)
• What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories [23.623074512572593]
  We aim to explore to what extent language models are capable of discerning among senses at inference time. We leverage the relation between word senses and domains, and cast Word Sense Disambiguation (WSD) as a textual entailment problem. Our results show that this approach is indeed effective, close to supervised systems.
  arXiv  Detail & Related papers  (2023-02-07T09:55:07Z)
• Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis [64.70116276295609]
  SentiWSP is a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks. SentiWSP achieves new state-of-the-art performance on various sentence-level and aspect-level sentiment classification benchmarks.
  arXiv  Detail & Related papers  (2022-10-18T12:25:29Z)
• Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings [11.475144702935568]
  We quantify how much the contextualized embeddings of each word sense vary across contexts in typical pre-trained models. We find that word representations are position-biased, where the first words in different contexts tend to be more similar.
  arXiv  Detail & Related papers  (2022-08-20T12:27:25Z)
• Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories [47.03271152494389]
  Word Sense Disambiguation aims to automatically identify the exact meaning of one word according to its context. Existing supervised models struggle to make correct predictions on rare word senses due to limited training data. We propose a gloss alignment algorithm that can align definition sentences with the same meaning from different sense inventories to collect rich lexical knowledge.
  arXiv  Detail & Related papers  (2021-10-27T00:04:33Z)
• Sense representations for Portuguese: experiments with sense embeddings and deep neural language models [0.0]
  Unsupervised sense representations can induce different senses of a word by analyzing its contextual semantics in a text. We present the first experiments carried out for generating sense embeddings for Portuguese.
  arXiv  Detail & Related papers  (2021-08-31T18:07:01Z)
• Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation [56.830395467247016]
  We propose a model of semantic memory for WSD in a meta-learning setting. Our model is based on hierarchical variational inference and incorporates an adaptive memory update rule via a hypernetwork. We show our model advances the state of the art in few-shot WSD, supports effective learning in extremely data scarce scenarios.
  arXiv  Detail & Related papers  (2021-06-05T20:40:01Z)
• Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge [0.0]
  We investigate whether recent advances in NLP, specifically contextualized word embeddings, capture human-like distinctions between English word senses. We find that participants' judgments of the relatedness between senses are correlated with distances between senses in the BERT embedding space. Our findings point towards the potential utility of continuous-space representations of sense meanings.
  arXiv  Detail & Related papers  (2020-10-25T07:56:52Z)
• Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders [79.38278330678965]
  A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense.
  arXiv  Detail & Related papers  (2020-05-06T04:21:45Z)
• Word Sense Disambiguation for 158 Languages using Word Embeddings Only [80.79437083582643]
  Disambiguation of word senses in context is easy for humans, but a major challenge for automatic approaches. We present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings.
  arXiv  Detail & Related papers  (2020-03-14T14:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.