Evaluating a Multi-sense Definition Generation Model for Multiple Languages

• URL: http://arxiv.org/abs/2006.07398v1
• Date: Fri, 12 Jun 2020 18:15:59 GMT
• Title: Evaluating a Multi-sense Definition Generation Model for Multiple Languages
• Authors: Arman Kabiri, Paul Cook
• Abstract summary: We propose a context-agnostic approach to definition modeling, based on multi-sense word embeddings. Our results demonstrate that our proposed multi-sense model outperforms a single-sense model on all fifteen datasets.
• Score: 1.5229257192293197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most prior work on definition modeling has not accounted for polysemy, or has done so by considering definition modeling for a target word in a given context. In contrast, in this study, we propose a context-agnostic approach to definition modeling, based on multi-sense word embeddings, that is capable of generating multiple definitions for a target word. In further, contrast to most prior work, which has primarily focused on English, we evaluate our proposed approach on fifteen different datasets covering nine languages from several language families. To evaluate our approach we consider several variations of BLEU. Our results demonstrate that our proposed multi-sense model outperforms a single-sense model on all fifteen datasets.

Related papers

• Benchmarking Concept-Spilling Across Languages in LLMs [7.577675422356702]
  Large Language Models (LLMs) exhibit remarkable cross-lingual abilities, yet often exhibit a systematic bias toward representations from other languages.<n>This paper presents a novel comparative framework for evaluating multilingual semantic robustness by measuring how models handle polysemous words across languages.
  arXiv  Detail & Related papers  (2026-01-18T19:28:26Z)
• Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units [29.79935180749153]
  This paper investigates the enhancement of reasoning capabilities in language models through token-level multi-model collaboration.<n>We introduce a distribution distance-based dynamic selection strategy (DDS) to optimize the multi-model collaboration process.
  arXiv  Detail & Related papers  (2025-08-26T07:41:33Z)
• Adapting Definition Modeling for New Languages: A Case Study on Belarusian [2.2120851074630177]
  We propose a novel dataset of 43,150 definitions in Belarusian.<n>Our experiments demonstrate that adapting a definition modeling systems requires minimal amounts of data, but that there currently are gaps in what automatic metrics do capture.
  arXiv  Detail & Related papers  (2025-07-13T08:35:23Z)
• Multilingual Definition Modeling [1.9409995498330783]
  We use monolingual dictionary data for four new languages (Spanish, French, Portuguese, and German)<n>We test the performance of pre-trained multilingual language models on definition modeling of monosemic words when finetuned on this data.<n>Results show that multilingual language models can perform on-pair with English but cannot leverage potential cross-lingual synergies.
  arXiv  Detail & Related papers  (2025-06-02T09:48:37Z)
• xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation [2.9998889086656586]
  We propose an adaptation methodology for Large Vision-Language Models trained on English language data to improve their performance. We introduce a benchmark to evaluate the effectiveness of multilingual and multimodal embedding models.
  arXiv  Detail & Related papers  (2025-03-12T12:04:05Z)
• P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs [84.24644520272835]
  Large language models (LLMs) showcase varied multilingual capabilities across tasks like translation, code generation, and reasoning. Previous assessments often limited their scope to fundamental natural language processing (NLP) or isolated capability-specific tasks. We present a pipeline for selecting available and reasonable benchmarks from massive ones, addressing the oversight in previous work regarding the utility of these benchmarks. We introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets.
  arXiv  Detail & Related papers  (2024-11-14T01:29:36Z)
• Exploring Multiple Strategies to Improve Multilingual Coreference Resolution in CorefUD [0.0]
  This paper presents our end-to-end neural coreference resolution system. We first establish strong baseline models, including monolingual and cross-lingual variations. We propose several extensions to enhance performance across diverse linguistic contexts.
  arXiv  Detail & Related papers  (2024-08-29T20:27:05Z)
• Evaluating Large Language Models on Controlled Generation Tasks [92.64781370921486]
  We present an extensive analysis of various benchmarks including a sentence planning benchmark with different granularities. After comparing large language models against state-of-the-start finetuned smaller models, we present a spectrum showing large language models falling behind, are comparable, or exceed the ability of smaller models.
  arXiv  Detail & Related papers  (2023-10-23T03:48:24Z)
• Multilingual Conceptual Coverage in Text-to-Image Models [98.80343331645626]
  "Conceptual Coverage Across Languages" (CoCo-CroLa) is a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns. For each model we can assess "conceptual coverage" of a given target language relative to a source language by comparing the population of images generated for a series of tangible nouns in the source language to the population of images generated for each noun under translation in the target language.
  arXiv  Detail & Related papers  (2023-06-02T17:59:09Z)
• Exploiting Correlations Between Contexts and Definitions with Multiple Definition Modeling [13.608157331662026]
  Single Definition Modeling (SDM) does not adequately model the correlations and patterns among different contexts and definitions of words. In this paper, we design a new task called Multiple Definition Modeling (MDM) that pool together all contexts and definition of target words. We demonstrate and analyze the benefits of MDM, including improving SDM's performance by using MDM as the pretraining task and its comparable performance in the zero-shot setting.
  arXiv  Detail & Related papers  (2023-05-24T04:38:29Z)
• Black-box language model explanation by context length probing [7.526153863886609]
  We present context length probing, a novel explanation technique for causal language models. The technique is model-agnostic and does not rely on access to model internals beyond computing token-level probabilities. We apply context length probing to large pre-trained language models and offer some initial analyses and insights.
  arXiv  Detail & Related papers  (2022-12-30T16:24:10Z)
• Language Model Cascades [72.18809575261498]
  Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
  arXiv  Detail & Related papers  (2022-07-21T07:35:18Z)
• Specializing Multilingual Language Models: An Empirical Study [50.7526245872855]
  Contextualized word representations from pretrained multilingual language models have become the de facto standard for addressing natural language tasks. For languages rarely or never seen by these models, directly using such models often results in suboptimal representation or use of data.
  arXiv  Detail & Related papers  (2021-06-16T18:13:55Z)
• Multi-Sense Language Modelling [19.396806939258806]
  We propose a language model which not only predicts the next word, but also its sense in context. This higher prediction granularity may be useful for end tasks such as assistive writing. For sense prediction, we utilise a Graph Attention Network, which encodes definitions and example uses of word senses.
  arXiv  Detail & Related papers  (2020-12-10T16:06:05Z)
• VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling [24.775371434410328]
  We tackle the task of definition modeling, where the goal is to learn to generate definitions of words and phrases. Existing approaches for this task are discriminative, combining distributional and lexical semantics in an implicit rather than direct way. We propose a generative model for the task, introducing a continuous latent variable to explicitly model the underlying relationship between a phrase used within a context and its definition.
  arXiv  Detail & Related papers  (2020-10-07T02:48:44Z)
• Linguistic Structure Guided Context Modeling for Referring Image Segmentation [61.701577239317785]
  We propose a "gather-propagate-distribute" scheme to model multimodal context by cross-modal interaction. Our LSCM module builds a Dependency Parsing Tree Word Graph (DPT-WG) which guides all the words to include valid multimodal context of the sentence.
  arXiv  Detail & Related papers  (2020-10-01T16:03:51Z)
• Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
  A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size. We propose a fully compositional output embedding layer for language models. To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
  arXiv  Detail & Related papers  (2020-09-24T07:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.