Leveraging Domain Agnostic and Specific Knowledge for Acronym Disambiguation

• URL: http://arxiv.org/abs/2107.00316v1
• Date: Thu, 1 Jul 2021 09:10:00 GMT
• Title: Leveraging Domain Agnostic and Specific Knowledge for Acronym Disambiguation
• Authors: Qiwei Zhong, Guanxiong Zeng, Danqing Zhu, Yang Zhang, Wangli Lin, Ben Chen, Jiayu Tang
• Abstract summary: Acronym disambiguation aims to find the correct meaning of an ambiguous acronym in a text. We propose a Hierarchical Dual-path BERT method coined hdBERT to capture the general fine-grained and high-level specific representations. With a widely adopted SciAD dataset contained 62,441 sentences, we investigate the effectiveness of hdBERT.
• Score: 5.766754189548904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An obstacle to scientific document understanding is the extensive use of acronyms which are shortened forms of long technical phrases. Acronym disambiguation aims to find the correct meaning of an ambiguous acronym in a given text. Recent efforts attempted to incorporate word embeddings and deep learning architectures, and achieved significant effects in this task. In general domains, kinds of fine-grained pretrained language models have sprung up, thanks to the largescale corpora which can usually be obtained through crowdsourcing. However, these models based on domain agnostic knowledge might achieve insufficient performance when directly applied to the scientific domain. Moreover, obtaining large-scale high-quality annotated data and representing high-level semantics in the scientific domain is challenging and expensive. In this paper, we consider both the domain agnostic and specific knowledge, and propose a Hierarchical Dual-path BERT method coined hdBERT to capture the general fine-grained and high-level specific representations for acronym disambiguation. First, the context-based pretrained models, RoBERTa and SciBERT, are elaborately involved in encoding these two kinds of knowledge respectively. Second, multiple layer perceptron is devised to integrate the dualpath representations simultaneously and outputs the prediction. With a widely adopted SciAD dataset contained 62,441 sentences, we investigate the effectiveness of hdBERT. The experimental results exhibit that the proposed approach outperforms state-of-the-art methods among various evaluation metrics. Specifically, its macro F1 achieves 93.73%.

Related papers

• Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendations [50.81844184210381]
  We propose a general paradigm that augments large language models with DOmain-specific KnowledgE to enhance their performance on practical applications, namely DOKE. This paradigm relies on a domain knowledge extractor, working in three steps: 1) preparing effective knowledge for the task; 2) selecting the knowledge for each specific sample; and 3) expressing the knowledge in an LLM-understandable way.
  arXiv  Detail & Related papers  (2023-11-16T07:09:38Z)
• Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding [46.00400830499326]
  We propose a Knowledge-enhanced lANGuAge Representation learning framework for various clOsed dOmains (KANGAROO) In the experiments, we evaluate KANGAROO over various knowledge-aware and general NLP tasks in both full and few-shot learning settings.
  arXiv  Detail & Related papers  (2023-11-12T07:37:24Z)
• Adapting Knowledge for Few-shot Table-to-Text Generation [35.59842534346997]
  We propose a novel framework: Adapt-Knowledge-to-Generate (AKG) AKG adapts unlabeled domain-specific knowledge into the model, which brings at least three benefits. Our model achieves superior performance in terms of both fluency and accuracy as judged by human and automatic evaluations.
  arXiv  Detail & Related papers  (2023-02-24T05:48:53Z)
• Language-aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification [15.842081807249416]
  It is necessary to explore the effectiveness of linguistic mode in assisting hyperspectral image classification. Large-scale pre-training image-text foundation models have demonstrated great performance in a variety of downstream applications. A Language-aware Domain Generalization Network (LDGnet) is proposed to learn cross-domain invariant representation.
  arXiv  Detail & Related papers  (2022-09-06T10:06:10Z)
• Prior Knowledge Guided Unsupervised Domain Adaptation [82.9977759320565]
  We propose a Knowledge-guided Unsupervised Domain Adaptation (KUDA) setting where prior knowledge about the target class distribution is available. In particular, we consider two specific types of prior knowledge about the class distribution in the target domain: Unary Bound and Binary Relationship. We propose a rectification module that uses such prior knowledge to refine model generated pseudo labels.
  arXiv  Detail & Related papers  (2022-07-18T18:41:36Z)
• Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis [96.53859361560505]
  We propose a knowledge graph augmented network (KGAN) to incorporate external knowledge with explicitly syntactic and contextual information. KGAN captures the sentiment feature representations from multiple perspectives, i.e., context-, syntax- and knowledge-based. Experiments on three popular ABSA benchmarks demonstrate the effectiveness and robustness of our KGAN.
  arXiv  Detail & Related papers  (2022-01-13T08:25:53Z)
• Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction [6.46964825569749]
  In this paper, we tackle a more challenging setup consisting of domain-specific corpora with much longer n-gram and highly specialized terms. To encourage span-level representations in generation, we additionally impose a source-sentence conditioned masked span prediction loss in the decoder. Experimental results on three domain-specific corpora in two language pairs demonstrate that the proposed training scheme can improve the performance of existing lexically constrained methods.
  arXiv  Detail & Related papers  (2021-05-12T08:11:33Z)
• What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation [74.42107665213909]
  Acronyms are the short forms of phrases that facilitate conveying lengthy sentences in documents and serve as one of the mainstays of writing. Due to their importance, identifying acronyms and corresponding phrases (AI) and finding the correct meaning of each acronym (i.e., acronym disambiguation (AD)) are crucial for text understanding. Despite the recent progress on this task, there are some limitations in the existing datasets which hinder further improvement.
  arXiv  Detail & Related papers  (2020-10-28T00:12:36Z)
• Text Recognition in Real Scenarios with a Few Labeled Samples [55.07859517380136]
  Scene text recognition (STR) is still a hot research topic in computer vision field. This paper proposes a few-shot adversarial sequence domain adaptation (FASDA) approach to build sequence adaptation. Our approach can maximize the character-level confusion between the source domain and the target domain.
  arXiv  Detail & Related papers  (2020-06-22T13:03:01Z)
• Domain Adaption for Knowledge Tracing [65.86619804954283]
  We propose a novel adaptable framework, namely knowledge tracing (AKT) to address the DAKT problem. For the first aspect, we incorporate the educational characteristics (e.g., slip, guess, question texts) based on the deep knowledge tracing (DKT) to obtain a good performed knowledge tracing model. For the second aspect, we propose and adopt three domain adaptation processes. First, we pre-train an auto-encoder to select useful source instances for target model training.
  arXiv  Detail & Related papers  (2020-01-14T15:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.