Partial Colexifications Improve Concept Embeddings

• URL: http://arxiv.org/abs/2502.09743v1
• Date: Thu, 13 Feb 2025 19:58:00 GMT
• Title: Partial Colexifications Improve Concept Embeddings
• Authors: Arne Rubehn, Johann-Mattis List,
• Abstract summary: We show how partial colexifications can be used to improve concept embeddings in meaningful ways. The learned embeddings are evaluated against lexical similarity ratings, recorded instances of semantic shift, and word association data.
• Score: 1.3351610617039973
• License:
• Abstract: While the embedding of words has revolutionized the field of Natural Language Processing, the embedding of concepts has received much less attention so far. A dense and meaningful representation of concepts, however, could prove useful for several tasks in computational linguistics, especially those involving cross-linguistic data or sparse data from low resource languages. First methods that have been proposed so far embed concepts from automatically constructed colexification networks. While these approaches depart from automatically inferred polysemies, attested across a larger number of languages, they are restricted to the word level, ignoring lexical relations that would only hold for parts of the words in a given language. Building on recently introduced methods for the inference of partial colexifications, we show how they can be used to improve concept embeddings in meaningful ways. The learned embeddings are evaluated against lexical similarity ratings, recorded instances of semantic shift, and word association data. We show that in all evaluation tasks, the inclusion of partial colexifications lead to improved concept representations and better results. Our results further show that the learned embeddings are able to capture and represent different semantic relationships between concepts.

Related papers

• Pixel Sentence Representation Learning [67.4775296225521]
  In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process. We employ visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to be perceived as continuous. Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision.
  arXiv  Detail & Related papers  (2024-02-13T02:46:45Z)
• Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation [100.81837601210597]
  We propose Concept Curation (CoCu) to bridge the gap between visual and textual semantics in pre-training data. CoCu achieves superb zero-shot transfer performance and greatly boosts language-supervised segmentation baseline by a large margin.
  arXiv  Detail & Related papers  (2023-09-24T00:05:39Z)
• A Comprehensive Empirical Evaluation of Existing Word Embedding Approaches [5.065947993017158]
  We present the characteristics of existing word embedding approaches and analyze them with regard to many classification tasks. Traditional approaches mostly use matrix factorization to produce word representations, and they are not able to capture the semantic and syntactic regularities of the language very well. On the other hand, Neural-network-based approaches can capture sophisticated regularities of the language and preserve the word relationships in the generated word representations.
  arXiv  Detail & Related papers  (2023-03-13T15:34:19Z)
• Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation [70.58243648754507]
  We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR) Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic similarity and transfer tasks.
  arXiv  Detail & Related papers  (2022-10-18T11:37:36Z)
• Visual Superordinate Abstraction for Robust Concept Learning [80.15940996821541]
  Concept learning constructs visual representations that are connected to linguistic semantics. We ascribe the bottleneck to a failure of exploring the intrinsic semantic hierarchy of visual concepts. We propose a visual superordinate abstraction framework for explicitly modeling semantic-aware visual subspaces.
  arXiv  Detail & Related papers  (2022-05-28T14:27:38Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• A comprehensive empirical analysis on cross-domain semantic enrichment for detection of depressive language [0.9749560288448115]
  We start with a rich word embedding pre-trained from a large general dataset, which is then augmented with embeddings learned from a much smaller and more specific domain dataset through a simple non-linear mapping mechanism. We show that our augmented word embedding representations achieve a significantly better F1 score than the others, specially when applied to a high quality dataset.
  arXiv  Detail & Related papers  (2021-06-24T07:15:09Z)
• Enhanced word embeddings using multi-semantic representation through lexical chains [1.8199326045904998]
  We propose two novel algorithms, called Flexible Lexical Chain II and Fixed Lexical Chain II. These algorithms combine the semantic relations derived from lexical chains, prior knowledge from lexical databases, and the robustness of the distributional hypothesis in word embeddings as building blocks forming a single system. Our results show the integration between lexical chains and word embeddings representations sustain state-of-the-art results, even against more complex systems.
  arXiv  Detail & Related papers  (2021-01-22T09:43:33Z)
• On the Learnability of Concepts: With Applications to Comparing Word Embedding Algorithms [0.0]
  We introduce the notion of "concept" as a list of words that have shared semantic content. We first use this notion to measure the learnability of concepts on pretrained word embeddings. We then develop a statistical analysis of concept learnability, based on hypothesis testing and ROC curves, in order to compare the relative merits of various embedding algorithms.
  arXiv  Detail & Related papers  (2020-06-17T14:25:36Z)
• On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
  We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings. We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
  arXiv  Detail & Related papers  (2020-04-09T19:50:32Z)
• A Common Semantic Space for Monolingual and Cross-Lingual Meta-Embeddings [10.871587311621974]
  This paper presents a new technique for creating monolingual and cross-lingual meta-embeddings. Existing word vectors are projected to a common semantic space using linear transformations and averaging. The resulting cross-lingual meta-embeddings also exhibit excellent cross-lingual transfer learning capabilities.
  arXiv  Detail & Related papers  (2020-01-17T15:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.