Related papers: Analyzing Semantic Change through Lexical Replacements

Analyzing Semantic Change through Lexical Replacements

URL: http://arxiv.org/abs/2404.18570v1
Date: Mon, 29 Apr 2024 10:20:41 GMT
Title: Analyzing Semantic Change through Lexical Replacements
Authors: Francesco Periti, Pierluigi Cassotti, Haim Dubossarsky, Nina Tahmasebi,
Abstract summary: We study the effect of unexpected contexts introduced by textitlexical replacements We propose a textitreplacement schema where a target word is substituted with lexical replacements of varying relatedness. We are the first to evaluate the use of LLaMa for semantic change detection.
Score: 2.509907053583601
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model \textit{semantic change} by studying the effect of unexpected contexts introduced by \textit{lexical replacements}. We propose a \textit{replacement schema} where a target word is substituted with lexical replacements of varying relatedness, thus simulating different kinds of semantic change. Furthermore, we leverage the replacement schema as a basis for a novel \textit{interpretable} model for semantic change. We are also the first to evaluate the use of LLaMa for semantic change detection.

Related papers

Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective [50.261681681643076]
We propose a novel metric called SemVarEffect and a benchmark named SemVarBench to evaluate the causality between semantic variations in inputs and outputs in text-to-image synthesis. Our work establishes an effective evaluation framework that advances the T2I synthesis community's exploration of human instruction understanding.
arXiv Detail & Related papers (2024-10-14T08:45:35Z)
Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types [1.3436368800886478]
We present a model that leverages information both from synchronic lexical relations and definitions of word meanings. Specifically, we use synset definitions and hierarchy information from WordNet and test it on a digitized version of Blank's (1997) dataset of semantic change types.
arXiv Detail & Related papers (2024-06-05T16:52:21Z)
Unsupervised Lexical Substitution with Decontextualised Embeddings [48.00929769805882]
We propose a new unsupervised method for lexical substitution using pre-trained language models. Our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines.
arXiv Detail & Related papers (2022-09-17T03:51:47Z)
Contextualized language models for semantic change detection: lessons learned [4.436724861363513]
We present a qualitative analysis of the outputs of contextualized embedding-based methods for detecting diachronic semantic change. Our findings show that contextualized methods can often predict high change scores for words which are not undergoing any real diachronic semantic shift. Our conclusion is that pre-trained contextualized language models are prone to confound changes in lexicographic senses and changes in contextual variance.
arXiv Detail & Related papers (2022-08-31T23:35:24Z)
LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution [76.615287796753]
We introduce LexSubCon, an end-to-end lexical substitution framework based on contextual embedding models. This is achieved by combining contextual information with knowledge from structured lexical resources. Our experiments show that LexSubCon outperforms previous state-of-the-art methods on LS07 and CoInCo benchmark datasets.
arXiv Detail & Related papers (2021-07-11T21:25:56Z)
Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks [58.87961226278285]
We propose a self-supervised approach to model lexical semantic change. We show that our method can be used for the detection of semantic change with any alignment method. We illustrate the utility of our techniques using experimental results on three different datasets.
arXiv Detail & Related papers (2021-01-30T18:59:43Z)
Lexical semantic change for Ancient Greek and Latin [61.69697586178796]
Associating a word's correct meaning in its historical context is a central challenge in diachronic research. We build on a recent computational approach to semantic change based on a dynamic Bayesian mixture model. We provide a systematic comparison of dynamic Bayesian mixture models for semantic change with state-of-the-art embedding-based models.
arXiv Detail & Related papers (2021-01-22T12:04:08Z)
Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
arXiv Detail & Related papers (2020-10-11T15:13:18Z)
Unsupervised Embedding-based Detection of Lexical Semantic Changes [1.7403133838762452]
This paper describes EmbLexChange, a system introduced by the "Life-Language" team for SemEval-2020 Task 1. EmmLexChange is defined as the divergence between the embedding based profiles of word w in the source and the target domains. We show that using a resampling framework for the selection of reference words, we can reliably detect lexical-semantic changes in English, German, Swedish, and Latin.
arXiv Detail & Related papers (2020-05-16T13:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.