A Unified Model for Reverse Dictionary and Definition Modelling

• URL: http://arxiv.org/abs/2205.04602v1
• Date: Mon, 9 May 2022 23:52:39 GMT
• Title: A Unified Model for Reverse Dictionary and Definition Modelling
• Authors: Pinzhen Chen, Zheng Zhao
• Abstract summary: We train a dual-way neural dictionary to guess words from definitions (reverse dictionary) and produce definitions given words (definition modelling) Our method learns the two tasks simultaneously, and handles unknown words via embeddings. It casts a word or a definition to the same representation space through a shared layer, then generates the other form from there, in a multi-task fashion.
• Score: 7.353994554197792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We train a dual-way neural dictionary to guess words from definitions (reverse dictionary), and produce definitions given words (definition modelling). Our method learns the two tasks simultaneously, and handles unknown words via embeddings. It casts a word or a definition to the same representation space through a shared layer, then generates the other form from there, in a multi-task fashion. The model achieves promising automatic scores without extra resources. Human annotators prefer the proposed model's outputs in both reference-less and reference-based evaluation, which indicates its practicality. Analysis suggests that multiple objectives benefit learning.

Related papers

• Domain Embeddings for Generating Complex Descriptions of Concepts in Italian Language [65.268245109828]
  We propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries. The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface. Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge.
  arXiv  Detail & Related papers  (2024-02-26T15:04:35Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models [77.45934004406283]
  We systematically study decompounding, the task of splitting compound words into their constituents. We introduce a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary. We introduce a novel methodology to train dedicated models for decompounding.
  arXiv  Detail & Related papers  (2023-05-23T16:32:27Z)
• IRB-NLP at SemEval-2022 Task 1: Exploring the Relationship Between Words and Their Semantic Representations [0.0]
  We present our findings based on the descriptive, exploratory, and predictive data analysis conducted on the CODWOE dataset. We give a detailed overview of the systems that we designed for Definition Modeling and Reverse Dictionary tasks.
  arXiv  Detail & Related papers  (2022-05-13T18:15:20Z)
• Dict-BERT: Enhancing Language Model Pre-training with Dictionary [42.0998323292348]
  Pre-trained language models (PLMs) aim to learn universal language representations by conducting self-supervised training tasks on large-scale corpora. In this work, we focus on enhancing language model pre-training by leveraging definitions of rare words in dictionaries. We propose two novel self-supervised pre-training tasks on word and sentence-level alignment between input text sequence and rare word definitions.
  arXiv  Detail & Related papers  (2021-10-13T04:29:14Z)
• PREDICT: Persian Reverse Dictionary [0.0]
  We compare four different architectures for implementing a Persian reverse dictionary (PREDICT) We evaluate our models using (phrase,word)Words extracted from the only Persian dictionaries available online. Experiments show that a model consisting of Long Short-Term Memory (LSTM) units enhanced by an additive attention mechanism is enough to produce suggestions comparable to (or in some cases better than) the word in the original dictionary.
  arXiv  Detail & Related papers  (2021-05-01T17:37:01Z)
• NLP-CIC @ DIACR-Ita: POS and Neighbor Based Distributional Models for Lexical Semantic Change in Diachronic Italian Corpora [62.997667081978825]
  We present our systems and findings on unsupervised lexical semantic change for the Italian language. The task is to determine whether a target word has evolved its meaning with time, only relying on raw-text from two time-specific datasets. We propose two models representing the target words across the periods to predict the changing words using threshold and voting schemes.
  arXiv  Detail & Related papers  (2020-11-07T11:27:18Z)
• VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling [24.775371434410328]
  We tackle the task of definition modeling, where the goal is to learn to generate definitions of words and phrases. Existing approaches for this task are discriminative, combining distributional and lexical semantics in an implicit rather than direct way. We propose a generative model for the task, introducing a continuous latent variable to explicitly model the underlying relationship between a phrase used within a context and its definition.
  arXiv  Detail & Related papers  (2020-10-07T02:48:44Z)
• Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions [87.33156149634392]
  We critically examine RefCOg, a standard benchmark for visual referring expression recognition. We show that 83.7% of test instances do not require reasoning on linguistic structure. We propose two methods, one based on contrastive learning and the other based on multi-task learning, to increase the robustness of ViLBERT.
  arXiv  Detail & Related papers  (2020-05-04T17:09:15Z)
• Lexical Sememe Prediction using Dictionary Definitions by Capturing Local Semantic Correspondence [94.79912471702782]
  Sememes, defined as the minimum semantic units of human languages, have been proven useful in many NLP tasks. We propose a Sememe Correspondence Pooling (SCorP) model, which is able to capture this kind of matching to predict sememes. We evaluate our model and baseline methods on a famous sememe KB HowNet and find that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-01-16T17:30:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.