PSG: Prompt-based Sequence Generation for Acronym Extraction

• URL: http://arxiv.org/abs/2111.14301v1
• Date: Mon, 29 Nov 2021 02:14:38 GMT
• Title: PSG: Prompt-based Sequence Generation for Acronym Extraction
• Authors: Bin Li, Fei Xia, Yixuan Weng, Xiusheng Huang, Bin Sun, Shutao Li
• Abstract summary: We propose a Prompt-based Sequence Generation (PSG) method for the acronym extraction task. Specifically, we design a template for prompting the extracted acronym texts with auto-regression. A position extraction algorithm is designed for extracting the position of the generated answers.
• Score: 26.896811663334162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Acronym extraction aims to find acronyms (i.e., short-forms) and their meanings (i.e., long-forms) from the documents, which is important for scientific document understanding (SDU@AAAI-22) tasks. Previous works are devoted to modeling this task as a paragraph-level sequence labeling problem. However, it lacks the effective use of the external knowledge, especially when the datasets are in a low-resource setting. Recently, the prompt-based method with the vast pre-trained language model can significantly enhance the performance of the low-resourced downstream tasks. In this paper, we propose a Prompt-based Sequence Generation (PSG) method for the acronym extraction task. Specifically, we design a template for prompting the extracted acronym texts with auto-regression. A position extraction algorithm is designed for extracting the position of the generated answers. The results on the acronym extraction of Vietnamese and Persian in a low-resource setting show that the proposed method outperforms all other competitive state-of-the-art (SOTA) methods.

Related papers

• Dense X Retrieval: What Retrieval Granularity Should We Use? [56.90827473115201]
  Often-overlooked design choice is the retrieval unit in which the corpus is indexed, e.g. document, passage, or sentence. We introduce a novel retrieval unit, proposition, for dense retrieval. Experiments reveal that indexing a corpus by fine-grained units such as propositions significantly outperforms passage-level units in retrieval tasks.
  arXiv  Detail & Related papers  (2023-12-11T18:57:35Z)
• RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation [61.53695868960846]
  We propose compressing retrieved documents into textual summaries prior to in-context integration. This not only reduces the computational costs but also relieves the burden of LMs to identify relevant information in long retrieved documents. We show that our compressors trained for one LM can transfer to other LMs on the language modeling task and provide summaries largely faithful to the retrieved documents.
  arXiv  Detail & Related papers  (2023-10-06T17:55:36Z)
• EntropyRank: Unsupervised Keyphrase Extraction via Side-Information Optimization for Language Model-based Text Compression [62.261476176242724]
  We propose an unsupervised method to extract keywords and keyphrases from texts based on a pre-trained language model (LM) and Shannon's information. Specifically, our method extracts phrases having the highest conditional entropy under the LM.
  arXiv  Detail & Related papers  (2023-08-25T14:23:40Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• Gradient Imitation Reinforcement Learning for General Low-Resource Information Extraction [80.64518530825801]
  We develop a Gradient Reinforcement Learning (GIRL) method to encourage pseudo-labeled data to imitate the gradient descent direction on labeled data. We also leverage GIRL to solve all IE sub-tasks (named entity recognition, relation extraction, and event extraction) in low-resource settings.
  arXiv  Detail & Related papers  (2022-11-11T05:37:19Z)
• Prompt-based Text Entailment for Low-Resource Named Entity Recognition [21.017890579840145]
  We propose Prompt-based Text Entailment (PTE) for low-resource named entity recognition. The proposed method achieves competitive performance on the CoNLL03 dataset.
  arXiv  Detail & Related papers  (2022-11-06T06:13:38Z)
• An Ensemble Approach to Acronym Extraction using Transformers [7.88595796865485]
  Acronyms are abbreviated units of a phrase constructed by using initial components of the phrase in a text. This paper discusses an ensemble approach for the task of Acronym Extraction.
  arXiv  Detail & Related papers  (2022-01-09T14:49:46Z)
• CABACE: Injecting Character Sequence Information and Domain Knowledge for Enhanced Acronym and Long-Form Extraction [0.0]
  We propose a novel framework CABACE: Character-Aware BERT for ACronym Extraction. It takes into account character sequences in text and is adapted to scientific and legal domains by masked language modelling. We show that the proposed framework is better suited than baseline models for zero-shot generalization to non-English languages.
  arXiv  Detail & Related papers  (2021-12-25T14:03:09Z)
• BERT-based Acronym Disambiguation with Multiple Training Strategies [8.82012912690778]
  Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. We propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.
  arXiv  Detail & Related papers  (2021-02-25T05:40:21Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)
• Extraction of Templates from Phrases Using Sequence Binary Decision Diagrams [3.867363075280544]
  This paper presents an unsupervised approach for extracting templates from only tagged text by using a novel relaxed variant of the Sequence Binary Decision Diagram (SeqBDD) The main contribution of this paper is a relaxed form of the SeqBDD construction algorithm that enables it to form general representations from a small amount of data. Experiments show that the method is capable of high-quality extraction on tasks based on verb+preposition templates from corpora and phrasal templates from short messages from social media.
  arXiv  Detail & Related papers  (2020-01-28T05:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.