Language models in word sense disambiguation for Polish

• URL: http://arxiv.org/abs/2111.13982v1
• Date: Sat, 27 Nov 2021 20:47:53 GMT
• Title: Language models in word sense disambiguation for Polish
• Authors: Agnieszka Mykowiecka, Agnieszka A. Mykowiecka, Piotr Rychlik
• Abstract summary: We use neural language models to predict words similar to those being disambiguated. On the basis of these words, we predict the partition of word senses in different ways.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In the paper, we test two different approaches to the {unsupervised} word sense disambiguation task for Polish. In both methods, we use neural language models to predict words similar to those being disambiguated and, on the basis of these words, we predict the partition of word senses in different ways. In the first method, we cluster selected similar words, while in the second, we cluster vectors representing their subsets. The evaluation was carried out on texts annotated with plWordNet senses and provided a relatively good result (F1=0.68 for all ambiguous words). The results are significantly better than those obtained for the neural model-based unsupervised method proposed in \cite{waw:myk:17:Sense} and are at the level of the supervised method presented there. The proposed method may be a way of solving word sense disambiguation problem for languages that lack sense annotated data.

Related papers

• Homonym Sense Disambiguation in the Georgian Language [49.1574468325115]
  This research proposes a novel approach to the Word Sense Disambiguation (WSD) task in the Georgian language. It is based on supervised fine-tuning of a pre-trained Large Language Model (LLM) on a dataset formed by filtering the Georgian Common Crawls corpus.
  arXiv  Detail & Related papers  (2024-04-24T21:48:43Z)
• Pixel Sentence Representation Learning [67.4775296225521]
  In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process. We employ visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to be perceived as continuous. Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision.
  arXiv  Detail & Related papers  (2024-02-13T02:46:45Z)
• DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning [59.4644086610381]
  We propose a novel denoising objective that inherits from another perspective, i.e., the intra-sentence perspective. By introducing both discrete and continuous noise, we generate noisy sentences and then train our model to restore them to their original form. Our empirical evaluations demonstrate that this approach delivers competitive results on both semantic textual similarity (STS) and a wide range of transfer tasks.
  arXiv  Detail & Related papers  (2024-01-24T17:48:45Z)
• Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization [2.9950872478176627]
  We study how the presence of a prior sentence can resolve relative clause ambiguities. Results show that a neurosymbolic, based on proof nets, is more open to data bias correction than an approach based on universal dependencies.
  arXiv  Detail & Related papers  (2023-05-24T09:04:18Z)
• CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models [77.45934004406283]
  We systematically study decompounding, the task of splitting compound words into their constituents. We introduce a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary. We introduce a novel methodology to train dedicated models for decompounding.
  arXiv  Detail & Related papers  (2023-05-23T16:32:27Z)
• Contextualized Word Vector-based Methods for Discovering Semantic Differences with No Training nor Word Alignment [17.229611956178818]
  We propose methods for discovering semantic differences in words appearing in two corpora. The key idea is that the coverage of meanings is reflected in the norm of its mean word vector. We show these advantages for native and non-native English corpora and also for historical corpora.
  arXiv  Detail & Related papers  (2023-05-19T08:27:17Z)
• Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks [84.61578555312288]
  We introduce a method for the prediction of disambiguation errors based on statistical data properties. We develop a simple adversarial attack strategy that minimally perturbs sentences in order to elicit disambiguation errors. Our findings indicate that disambiguation robustness varies substantially between domains and that different models trained on the same data are vulnerable to different attacks.
  arXiv  Detail & Related papers  (2020-11-03T17:01:44Z)
• Neural disambiguation of lemma and part of speech in morphologically rich languages [0.6346772579930928]
  We consider the problem of disambiguating the lemma and part of speech of ambiguous words in morphologically rich languages. We propose a method for disambiguating ambiguous words in context, using a large un-annotated corpus of text, and a morphological analyser.
  arXiv  Detail & Related papers  (2020-07-12T21:48:52Z)
• Combining Neural Language Models for WordSense Induction [0.5199765487172326]
  Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context. In this work, we apply this approach to the Russian language and improve it in two ways.
  arXiv  Detail & Related papers  (2020-06-23T17:57:25Z)
• Comparative Analysis of Word Embeddings for Capturing Word Similarities [0.0]
  Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans.
  arXiv  Detail & Related papers  (2020-05-08T01:16:03Z)
• Word Sense Disambiguation for 158 Languages using Word Embeddings Only [80.79437083582643]
  Disambiguation of word senses in context is easy for humans, but a major challenge for automatic approaches. We present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings.
  arXiv  Detail & Related papers  (2020-03-14T14:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.