The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage

• URL: http://arxiv.org/abs/2606.16560v1
• Date: Mon, 15 Jun 2026 11:03:39 GMT
• Title: The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage
• Abstract summary: We introduce two complementary benchmark datasets.<n>The BD-LSC dataset captures sense gain, sense loss, and stability across three time periods.<n>The ST-WSD dataset provides fine-grained, instance-level sense annotations for words combining slang and standard usages.
• Score: 20.764165408679762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic semantic change detection aims to identify how word meanings shift over time, offering insights into both linguistic and societal change. Despite recent progress in computational lexical semantic change (LSC), existing benchmarks and methods struggle to capture bi-directional semantic change, particularly cases where words simultaneously gain and lose senses. This problem is especially challenging for words that have both slang and standard meanings. To address these gaps, we introduce two complementary benchmark datasets. The Bi-Directional Lexical Semantic Change (BD-LSC) dataset captures sense gain, sense loss, and stability across three time periods, enabling the study of complex semantic trajectories. The SlangTrack Word Sense Disambiguation (ST-WSD) dataset provides fine-grained, instance-level sense annotations for words combining slang and standard usages, supporting systematic benchmarking of WSD and semantic change detection models. Using these benchmarks, we systematically evaluate models across different methodological families: unsupervised clustering using contextualised embeddings, supervised machine learning, transformer-based models, and state-of-the-art large language models. Among the evaluated systems, the few-shot GPT-4o model achieved the strongest aggregate performance on Exact Sense Match (ESM) and multi-label accuracy; however, Macro-F1 scores near 0.5 across all systems show that rare slang senses remain difficult, which we identify as the central open challenge.

Related papers

• Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective [50.261681681643076]
  We propose a novel metric called SemVarEffect and a benchmark named SemVarBench to evaluate the causality between semantic variations in inputs and outputs in text-to-image synthesis.<n>Our work establishes an effective evaluation framework that advances the T2I synthesis community's exploration of human instruction understanding.
  arXiv  Detail & Related papers  (2024-10-14T08:45:35Z)
• Graph-based Clustering for Detecting Semantic Change Across Time and Languages [10.058655884092094]
  We propose a graph-based clustering approach to capture nuanced changes in both high- and low-frequency word senses across time and languages. Our approach substantially surpasses previous approaches in the SemEval 2020 binary classification task across four languages.
  arXiv  Detail & Related papers  (2024-02-01T21:27:19Z)
• Co-Driven Recognition of Semantic Consistency via the Fusion of Transformer and HowNet Sememes Knowledge [6.184249194474601]
  This paper proposes a co-driven semantic consistency recognition method based on the fusion of Transformer and HowNet sememes knowledge. BiLSTM is exploited to encode the conceptual semantic information and infer the semantic consistency.
  arXiv  Detail & Related papers  (2023-02-21T09:53:19Z)
• Word Sense Induction with Hierarchical Clustering and Mutual Information Maximization [14.997937028599255]
  Word sense induction is a difficult problem in natural language processing. We propose a novel unsupervised method based on hierarchical clustering and invariant information clustering. We empirically demonstrate that, in certain cases, our approach outperforms prior WSI state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-10-11T13:04:06Z)
• Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution [124.99894592871385]
  We present a large-scale comparative study of lexical substitution methods employing both old and most recent language models. We show that already competitive results achieved by SOTA LMs/MLMs can be further substantially improved if information about the target word is injected properly.
  arXiv  Detail & Related papers  (2022-06-07T16:16:19Z)
• Exploring Multi-Modal Representations for Ambiguity Detection & Coreference Resolution in the SIMMC 2.0 Challenge [60.616313552585645]
  We present models for effective Ambiguity Detection and Coreference Resolution in Conversational AI. Specifically, we use TOD-BERT and LXMERT based models, compare them to a number of baselines and provide ablation experiments. Our results show that (1) language models are able to exploit correlations in the data to detect ambiguity; and (2) unimodal coreference resolution models can avoid the need for a vision component.
  arXiv  Detail & Related papers  (2022-02-25T12:10:02Z)
• Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks [58.87961226278285]
  We propose a self-supervised approach to model lexical semantic change. We show that our method can be used for the detection of semantic change with any alignment method. We illustrate the utility of our techniques using experimental results on three different datasets.
  arXiv  Detail & Related papers  (2021-01-30T18:59:43Z)
• SChME at SemEval-2020 Task 1: A Model Ensemble for Detecting Lexical Semantic Change [58.87961226278285]
  This paper describes SChME, a method used in SemEval-2020 Task 1 on unsupervised detection of lexical semantic change. SChME usesa model ensemble combining signals of distributional models (word embeddings) and wordfrequency models where each model casts a vote indicating the probability that a word sufferedsemantic change according to that feature.
  arXiv  Detail & Related papers  (2020-12-02T23:56:34Z)
• SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces [63.17308641484404]
  We propose to identify clusters among different occurrences of each target word, considering these as representatives of different word meanings. Disagreements in obtained clusters naturally allow to quantify the level of semantic shift per each target word in four target languages. Our approach performs well both measured separately (per language) and overall, where we surpass all provided SemEval baselines.
  arXiv  Detail & Related papers  (2020-10-02T08:38:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.