Combining Neural Language Models for WordSense Induction
- URL: http://arxiv.org/abs/2006.13200v1
- Date: Tue, 23 Jun 2020 17:57:25 GMT
- Title: Combining Neural Language Models for WordSense Induction
- Authors: Nikolay Arefyev, Boris Sheludko, and Tatiana Aleksashina
- Abstract summary: Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word.
Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context.
In this work, we apply this approach to the Russian language and improve it in two ways.
- Score: 0.5199765487172326
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Word sense induction (WSI) is the problem of grouping occurrences of an
ambiguous word according to the expressed sense of this word. Recently a new
approach to this task was proposed, which generates possible substitutes for
the ambiguous word in a particular context using neural language models, and
then clusters sparse bag-of-words vectors built from these substitutes. In this
work, we apply this approach to the Russian language and improve it in two
ways. First, we propose methods of combining left and right contexts, resulting
in better substitutes generated. Second, instead of fixed number of clusters
for all ambiguous words we propose a technique for selecting individual number
of clusters for each word. Our approach established new state-of-the-art level,
improving current best results of WSI for the Russian language on two RUSSE
2018 datasets by a large margin.
Related papers
- Graph-based Clustering for Detecting Semantic Change Across Time and
Languages [10.058655884092094]
We propose a graph-based clustering approach to capture nuanced changes in both high- and low-frequency word senses across time and languages.
Our approach substantially surpasses previous approaches in the SemEval 2020 binary classification task across four languages.
arXiv Detail & Related papers (2024-02-01T21:27:19Z) - CompoundPiece: Evaluating and Improving Decompounding Performance of
Language Models [77.45934004406283]
We systematically study decompounding, the task of splitting compound words into their constituents.
We introduce a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary.
We introduce a novel methodology to train dedicated models for decompounding.
arXiv Detail & Related papers (2023-05-23T16:32:27Z) - Beyond Contrastive Learning: A Variational Generative Model for
Multilingual Retrieval [109.62363167257664]
We propose a generative model for learning multilingual text embeddings.
Our model operates on parallel data in $N$ languages.
We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
arXiv Detail & Related papers (2022-12-21T02:41:40Z) - Word Sense Induction with Hierarchical Clustering and Mutual Information
Maximization [14.997937028599255]
Word sense induction is a difficult problem in natural language processing.
We propose a novel unsupervised method based on hierarchical clustering and invariant information clustering.
We empirically demonstrate that, in certain cases, our approach outperforms prior WSI state-of-the-art methods.
arXiv Detail & Related papers (2022-10-11T13:04:06Z) - Always Keep your Target in Mind: Studying Semantics and Improving
Performance of Neural Lexical Substitution [124.99894592871385]
We present a large-scale comparative study of lexical substitution methods employing both old and most recent language models.
We show that already competitive results achieved by SOTA LMs/MLMs can be further substantially improved if information about the target word is injected properly.
arXiv Detail & Related papers (2022-06-07T16:16:19Z) - Language models in word sense disambiguation for Polish [0.0]
We use neural language models to predict words similar to those being disambiguated.
On the basis of these words, we predict the partition of word senses in different ways.
arXiv Detail & Related papers (2021-11-27T20:47:53Z) - Fake it Till You Make it: Self-Supervised Semantic Shifts for
Monolingual Word Embedding Tasks [58.87961226278285]
We propose a self-supervised approach to model lexical semantic change.
We show that our method can be used for the detection of semantic change with any alignment method.
We illustrate the utility of our techniques using experimental results on three different datasets.
arXiv Detail & Related papers (2021-01-30T18:59:43Z) - Improving Chinese Segmentation-free Word Embedding With Unsupervised
Association Measure [3.9435648520559177]
segmentation-free word embedding model is proposed by collecting n-grams vocabulary via a novel unsupervised association measure called pointwise association with times information(PATI)
The proposed method leverages more latent information from the corpus and thus is able to collect more valid n-grams that have stronger cohesion as embedding targets in unsegmented language data, such as Chinese texts.
arXiv Detail & Related papers (2020-07-05T13:55:19Z) - A Comparative Study of Lexical Substitution Approaches based on Neural
Language Models [117.96628873753123]
We present a large-scale comparative study of popular neural language and masked language models.
We show that already competitive results achieved by SOTA LMs/MLMs can be further improved if information about the target word is injected properly.
arXiv Detail & Related papers (2020-05-29T18:43:22Z) - Robust Cross-lingual Embeddings from Parallel Sentences [65.85468628136927]
We propose a bilingual extension of the CBOW method which leverages sentence-aligned corpora to obtain robust cross-lingual word representations.
Our approach significantly improves crosslingual sentence retrieval performance over all other approaches.
It also achieves parity with a deep RNN method on a zero-shot cross-lingual document classification task.
arXiv Detail & Related papers (2019-12-28T16:18:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.