BERT-based Acronym Disambiguation with Multiple Training Strategies

• URL: http://arxiv.org/abs/2103.00488v2
• Date: Tue, 2 Mar 2021 10:36:11 GMT
• Title: BERT-based Acronym Disambiguation with Multiple Training Strategies
• Authors: Chunguang Pan, Bingyan Song, Shengguang Wang and Zhipeng Luo
• Abstract summary: Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. We propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.
• Score: 8.82012912690778
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. Although it is convenient to use acronyms, sometimes they could be difficult to understand. Identifying the appropriate expansions of an acronym is a practical task in natural language processing. Since few works have been done for AD in scientific field, we propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection, task adaptive pretraining, adversarial training and pseudo labeling in this paper. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.

Related papers

• Automated Few-shot Classification with Instruction-Finetuned Language Models [76.69064714392165]
  We show that AuT-Few outperforms state-of-the-art few-shot learning methods. We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
  arXiv  Detail & Related papers  (2023-05-21T21:50:27Z)
• Unified Demonstration Retriever for In-Context Learning [56.06473069923567]
  Unified Demonstration Retriever (textbfUDR) is a single model to retrieve demonstrations for a wide range of tasks. We propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates. Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines.
  arXiv  Detail & Related papers  (2023-05-07T16:07:11Z)
• Disambiguation of Company names via Deep Recurrent Networks [101.90357454833845]
  We propose a Siamese LSTM Network approach to extract -- via supervised learning -- an embedding of company name strings. We analyse how an Active Learning approach to prioritise the samples to be labelled leads to a more efficient overall learning pipeline.
  arXiv  Detail & Related papers  (2023-03-07T15:07:57Z)
• How Does In-Context Learning Help Prompt Tuning? [55.78535874154915]
  Fine-tuning large language models is becoming ever more impractical due to their rapidly-growing scale. This motivates the use of parameter-efficient adaptation methods such as prompt tuning (PT), which adds a small number of tunable embeddings to an otherwise frozen model. Recently, Singhal et al. (2022) propose instruction prompt tuning'' (IPT), which combines PT with ICL by concatenating a natural language demonstration with learned prompt embeddings.
  arXiv  Detail & Related papers  (2023-02-22T17:45:12Z)
• SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation [26.896811663334162]
  We propose a Contrastive Learning of Acronym Disambiguation (SimCLAD) method to better understand the acronym meanings. The results on the acronym disambiguation of the scientific domain in English show that the proposed method outperforms all other competitive state-of-the-art (SOTA) methods.
  arXiv  Detail & Related papers  (2021-11-29T02:39:59Z)
• PSG: Prompt-based Sequence Generation for Acronym Extraction [26.896811663334162]
  We propose a Prompt-based Sequence Generation (PSG) method for the acronym extraction task. Specifically, we design a template for prompting the extracted acronym texts with auto-regression. A position extraction algorithm is designed for extracting the position of the generated answers.
  arXiv  Detail & Related papers  (2021-11-29T02:14:38Z)
• CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems [56.302581679816775]
  This paper proposes Comprehensive Instruction (CINS) that exploits PLMs with task-specific instructions. We design a schema (definition, constraint, prompt) of instructions and their customized realizations for three important downstream tasks in ToD. Experiments are conducted on these ToD tasks in realistic few-shot learning scenarios with small validation data.
  arXiv  Detail & Related papers  (2021-09-10T03:23:06Z)
• AT-BERT: Adversarial Training BERT for Acronym Identification Winning Solution for SDU@AAAI-21 [5.478126869836199]
  Acronym identification focuses on finding the acronyms and the phrases that have been abbreviated. Recent breakthroughs of language models pre-trained on large corpora clearly show that unsupervised pre-training can vastly improve the performance of downstream tasks. We present an Adversarial Training BERT method named AT-BERT, our winning solution to acronym identification task for Scientific Document Understanding (SDU) Challenge of AAAI 2021.
  arXiv  Detail & Related papers  (2021-01-11T05:02:34Z)
• Acronym Identification and Disambiguation Shared Tasks for Scientific Document Understanding [41.63345823743157]
  Acronyms are short forms of longer phrases frequently used in writing. Every text understanding tool should be capable of recognizing acronyms in text. To push forward research in this direction, we have organized two shared task for acronym identification and acronym disambiguation in scientific documents.
  arXiv  Detail & Related papers  (2020-12-22T00:29:15Z)
• Primer AI's Systems for Acronym Identification and Disambiguation [0.0]
  We introduce new methods for acronym identification and disambiguation. Our systems achieve significant performance gains over previously suggested methods. Both of our systems perform competitively on the SDU@AAAI-21 shared task leaderboard.
  arXiv  Detail & Related papers  (2020-12-14T23:59:05Z)
• What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation [74.42107665213909]
  Acronyms are the short forms of phrases that facilitate conveying lengthy sentences in documents and serve as one of the mainstays of writing. Due to their importance, identifying acronyms and corresponding phrases (AI) and finding the correct meaning of each acronym (i.e., acronym disambiguation (AD)) are crucial for text understanding. Despite the recent progress on this task, there are some limitations in the existing datasets which hinder further improvement.
  arXiv  Detail & Related papers  (2020-10-28T00:12:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.