Word-Embeddings Distinguish Denominal and Root-Derived Verbs in Semitic

• URL: http://arxiv.org/abs/2208.05721v1
• Date: Thu, 11 Aug 2022 09:31:37 GMT
• Title: Word-Embeddings Distinguish Denominal and Root-Derived Verbs in Semitic
• Authors: Ido Benbaji (MIT), Omri Doron (MIT), Ad\`ele H\'enot-Mortier (MIT)
• Abstract summary: We propose to test the validity of the two-level hypothesis in the context of Hebrew word embeddings. If the two-level hypothesis is borne out, we expect state-of-the-art Hebrew word embeddings to encode (1) a noun, (2) a denominal derived from it (via an upper-level operation), and (3) a verb related to the noun. We report that this hypothesis is verified by four embedding models of Hebrew: fastText, GloVe, Word2Vec and AlephBERT.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Proponents of the Distributed Morphology framework have posited the existence of two levels of morphological word formation: a lower one, leading to loose input-output semantic relationships; and an upper one, leading to tight input-output semantic relationships. In this work, we propose to test the validity of this assumption in the context of Hebrew word embeddings. If the two-level hypothesis is borne out, we expect state-of-the-art Hebrew word embeddings to encode (1) a noun, (2) a denominal derived from it (via an upper-level operation), and (3) a verb related to the noun (via a lower-level operation on the noun's root), in such a way that the denominal (2) should be closer in the embedding space to the noun (1) than the related verb (3) is to the same noun (1). We report that this hypothesis is verified by four embedding models of Hebrew: fastText, GloVe, Word2Vec and AlephBERT. This suggests that word embedding models are able to capture complex and fine-grained semantic properties that are morphologically motivated.

Related papers

• Agentivit\`a e telicit\`a in GilBERTo: implicazioni cognitive [77.71680953280436]
  The goal of this study is to investigate whether a Transformer-based neural language model infers lexical semantics. The semantic properties considered are telicity (also combined with definiteness) and agentivity.
  arXiv  Detail & Related papers  (2023-07-06T10:52:22Z)
• Unsupervised Mapping of Arguments of Deverbal Nouns to Their Corresponding Verbal Labels [52.940886615390106]
  Deverbal nouns are verbs commonly used in written English texts to describe events or actions, as well as their arguments. The solutions that do exist for handling arguments of nominalized constructions are based on semantic annotation. We propose to adopt a more syntactic approach, which maps the arguments of deverbal nouns to the corresponding verbal construction.
  arXiv  Detail & Related papers  (2023-06-24T10:07:01Z)
• A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces [11.564029462243631]
  We study semantic construal in grammatical constructions using large language models. We show that a word in subject position is interpreted as more agentive than the very same word in object position. Our method can probe the distributional meaning of syntactic constructions at a templatic level, abstracted away from specific lexemes.
  arXiv  Detail & Related papers  (2023-05-29T20:30:38Z)
• Not wacky vs. definitely wacky: A study of scalar adverbs in pretrained language models [0.0]
  Modern pretrained language models, such as BERT, RoBERTa and GPT-3 hold the promise of performing better on logical tasks than classic static word embeddings. We investigate the extent to which BERT, RoBERTa, GPT-2 and GPT-3 exhibit general, human-like, knowledge of these common words. We find that despite capturing some aspects of logical meaning, the models fall far short of human performance.
  arXiv  Detail & Related papers  (2023-05-25T18:56:26Z)
• The Causal Structure of Semantic Ambiguities [0.0]
  We identify two features: (1) joint plausibility degrees of different possible interpretations, and (2) causal structures according to which certain words play a more substantial role in the processes. We applied this theory to a dataset of ambiguous phrases extracted from Psycholinguistics literature and their human plausibility collected by us.
  arXiv  Detail & Related papers  (2022-06-14T12:56:34Z)
• Latent Topology Induction for Understanding Contextualized Representations [84.7918739062235]
  We study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models. We show there exists a network of latent states that summarize linguistic properties of contextualized representations.
  arXiv  Detail & Related papers  (2022-06-03T11:22:48Z)
• Decomposing lexical and compositional syntax and semantics with deep language models [82.81964713263483]
  The activations of language transformers like GPT2 have been shown to linearly map onto brain activity during speech comprehension. Here, we propose a taxonomy to factorize the high-dimensional activations of language models into four classes: lexical, compositional, syntactic, and semantic representations. The results highlight two findings. First, compositional representations recruit a more widespread cortical network than lexical ones, and encompass the bilateral temporal, parietal and prefrontal cortices.
  arXiv  Detail & Related papers  (2021-03-02T10:24:05Z)
• Topology of Word Embeddings: Singularities Reflect Polysemy [68.8204255655161]
  We introduce a topological measure of polysemy based on persistent homology that correlates well with the actual number of meanings of a word. We propose a simple, topologically motivated solution to the SemEval-2010 task on Word Sense Induction & Disambiguation.
  arXiv  Detail & Related papers  (2020-11-18T17:21:51Z)
• Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model [66.84264870118723]
  We present the first purely corpus-driven model of multi-lingual adjective ordering in the form of a latent-variable model. We provide strong converging evidence for the existence of universal, cross-linguistic, hierarchical adjective ordering tendencies.
  arXiv  Detail & Related papers  (2020-10-09T18:27:55Z)
• Using Holographically Compressed Embeddings in Question Answering [0.0]
  This research employs holographic compression of pre-trained embeddings to represent a token, its part-of-speech, and named entity type. The implementation, in a modified question answering recurrent deep learning network, shows that semantic relationships are preserved, and yields strong performance.
  arXiv  Detail & Related papers  (2020-07-14T18:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.