Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis

• URL: http://arxiv.org/abs/2305.11993v2
• Date: Tue, 25 Jul 2023 11:50:48 GMT
• Title: Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis
• Authors: Mario Giulianelli, Iris Luden, Raquel Fernandez, Andrey Kutuzov
• Abstract summary: We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable.
• Score: 3.515619810213763
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. Given a collection of usage examples for a target word, and the corresponding data-driven usage clusters (i.e., word senses), a definition is generated for each usage with a specialised Flan-T5 language model, and the most prototypical definition in a usage cluster is chosen as the sense label. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable, and how they can allow users -- historical linguists, lexicographers, or social scientists -- to explore and intuitively explain diachronic trajectories of word meaning. Semantic change analysis is only one of many possible applications of the `definitions as representations' paradigm. Beyond being human-readable, contextualised definitions also outperform token or usage sentence embeddings in word-in-context semantic similarity judgements, making them a new promising type of lexical representation for NLP.

Related papers

• Definition generation for lexical semantic change detection [3.7297237438000788]
  We use contextualized word definitions generated by large language models as semantic representations in the task of diachronic lexical semantic change detection (LSCD) In short, generated definitions are used as senses', and the change score of a target word is retrieved by comparing their distributions in two time periods under comparison. Our approach is on par with or outperforms prior non-supervised LSCD methods.
  arXiv  Detail & Related papers  (2024-06-20T10:13:08Z)
• Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types [1.3436368800886478]
  We present a model that leverages information both from synchronic lexical relations and definitions of word meanings. Specifically, we use synset definitions and hierarchy information from WordNet and test it on a digitized version of Blank's (1997) dataset of semantic change types.
  arXiv  Detail & Related papers  (2024-06-05T16:52:21Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings [17.803726860514193]
  Detection of semantic variation of words is an important task for various NLP applications. We argue that mean representations alone cannot accurately capture such semantic variations. We propose a method that uses the entire cohort of the contextualised embeddings of the target word.
  arXiv  Detail & Related papers  (2023-05-15T13:58:21Z)
• Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures [104.32063681736349]
  We present an approach to describe predicate-argument structures using natural language definitions instead of discrete labels. Our experiments and analyses on PropBank-style and FrameNet-style, dependency-based and span-based SRL also demonstrate that a flexible model with an interpretable output does not necessarily come at the expense of performance.
  arXiv  Detail & Related papers  (2022-12-02T11:19:16Z)
• On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
  We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar. We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods. Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
  arXiv  Detail & Related papers  (2021-10-15T21:41:16Z)
• Grammatical Profiling for Semantic Change Detection [6.3596637237946725]
  We use grammatical profiling as an alternative method for semantic change detection. We demonstrate that it can be used for semantic change detection and even outperforms some distributional semantic methods.
  arXiv  Detail & Related papers  (2021-09-21T18:38:18Z)
• Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks [58.87961226278285]
  We propose a self-supervised approach to model lexical semantic change. We show that our method can be used for the detection of semantic change with any alignment method. We illustrate the utility of our techniques using experimental results on three different datasets.
  arXiv  Detail & Related papers  (2021-01-30T18:59:43Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)
• Analysing Lexical Semantic Change with Contextualised Word Representations [7.071298726856781]
  We propose a novel method that exploits the BERT neural language model to obtain representations of word usages. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements.
  arXiv  Detail & Related papers  (2020-04-29T12:18:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.