SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation

• URL: http://arxiv.org/abs/2111.14306v1
• Date: Mon, 29 Nov 2021 02:39:59 GMT
• Title: SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation
• Authors: Bin Li, Fei Xia, Yixuan Weng, Xiusheng Huang, Bin Sun, Shutao Li
• Abstract summary: We propose a Contrastive Learning of Acronym Disambiguation (SimCLAD) method to better understand the acronym meanings. The results on the acronym disambiguation of the scientific domain in English show that the proposed method outperforms all other competitive state-of-the-art (SOTA) methods.
• Score: 26.896811663334162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Acronym disambiguation means finding the correct meaning of an ambiguous acronym in a given sentence from the dictionary, which is one of the key points for scientific document understanding (SDU@AAAI-22). Recently, many attempts have tried to solve this problem via fine-tuning the pre-trained masked language models (MLMs) in order to obtain a better acronym representation. However, the acronym meaning is varied under different contexts, whose corresponded sentence representation is the anisotropic distribution occupied with a narrow subset of the entire representation space. Such representations from pre-trained MLMs are not ideal for the acronym disambiguation from the given dictionary. In this paper, we propose a Simple framework for Contrastive Learning of Acronym Disambiguation (SimCLAD) method to better understand the acronym meanings. Specifically, we design a novel continual contrastive pre-training method that enhances the pre-trained model's generalization ability by learning the isotropic and discriminative distribution of the acronym sentence representations. The results on the acronym disambiguation of the scientific domain in English show that the proposed method outperforms all other competitive state-of-the-art (SOTA) methods.

Related papers

• On Translating Technical Terminology: A Translation Workflow for Machine-Translated Acronyms [3.053989095162017]
  We find that an important step is being missed: the translation of technical terms, specifically acronyms. Some state-of-the art machine translation systems like Google Translate which are publicly available can be erroneous when dealing with acronyms. We propose an additional step to the SL-TL (FR-EN) translation workflow where we first offer a new acronym corpus for public consumption and then experiment with a search-based thresholding algorithm.
  arXiv  Detail & Related papers  (2024-09-26T15:18:34Z)
• Analyzing the Role of Semantic Representations in the Era of Large Language Models [104.18157036880287]
  We investigate the role of semantic representations in the era of large language models (LLMs) We propose an AMR-driven chain-of-thought prompting method, which we call AMRCoT. We find that it is difficult to predict which input examples AMR may help or hurt on, but errors tend to arise with multi-word expressions.
  arXiv  Detail & Related papers  (2024-05-02T17:32:59Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery [24.45800271294178]
  Key challenges lie in pseudo label disambiguation and representation learning. We propose a decoupled prototype learning framework (DPL) to decouple pseudo label disambiguation and representation learning. Experiments and analysis on three benchmark datasets show the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-05-28T12:01:34Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Alleviating Over-smoothing for Unsupervised Sentence Representation [96.19497378628594]
  We present a Simple method named Self-Contrastive Learning (SSCL) to alleviate this issue. Our proposed method is quite simple and can be easily extended to various state-of-the-art models for performance boosting.
  arXiv  Detail & Related papers  (2023-05-09T11:00:02Z)
• PSG: Prompt-based Sequence Generation for Acronym Extraction [26.896811663334162]
  We propose a Prompt-based Sequence Generation (PSG) method for the acronym extraction task. Specifically, we design a template for prompting the extracted acronym texts with auto-regression. A position extraction algorithm is designed for extracting the position of the generated answers.
  arXiv  Detail & Related papers  (2021-11-29T02:14:38Z)
• BERT-based Acronym Disambiguation with Multiple Training Strategies [8.82012912690778]
  Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. We propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.
  arXiv  Detail & Related papers  (2021-02-25T05:40:21Z)
• Accurate Word Representations with Universal Visual Guidance [55.71425503859685]
  This paper proposes a visual representation method to explicitly enhance conventional word embedding with multiple-aspect senses from visual guidance. We build a small-scale word-image dictionary from a multimodal seed dataset where each word corresponds to diverse related images. Experiments on 12 natural language understanding and machine translation tasks further verify the effectiveness and the generalization capability of the proposed approach.
  arXiv  Detail & Related papers  (2020-12-30T09:11:50Z)
• Primer AI's Systems for Acronym Identification and Disambiguation [0.0]
  We introduce new methods for acronym identification and disambiguation. Our systems achieve significant performance gains over previously suggested methods. Both of our systems perform competitively on the SDU@AAAI-21 shared task leaderboard.
  arXiv  Detail & Related papers  (2020-12-14T23:59:05Z)
• What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation [74.42107665213909]
  Acronyms are the short forms of phrases that facilitate conveying lengthy sentences in documents and serve as one of the mainstays of writing. Due to their importance, identifying acronyms and corresponding phrases (AI) and finding the correct meaning of each acronym (i.e., acronym disambiguation (AD)) are crucial for text understanding. Despite the recent progress on this task, there are some limitations in the existing datasets which hinder further improvement.
  arXiv  Detail & Related papers  (2020-10-28T00:12:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.