A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks

• URL: http://arxiv.org/abs/2212.05251v2
• Date: Mon, 5 Jun 2023 08:14:47 GMT
• Title: A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks
• Authors: Ruiqing Ding, Xiao Han, Leye Wang
• Abstract summary: We propose KnowledgeDA, a unified domain language model development service to enhance the task-specific training procedure with domain knowledge graphs. We implement a prototype of KnowledgeDA to learn language models for two domains, healthcare and software development.
• Score: 10.28161912127425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: By focusing the pre-training process on domain-specific corpora, some domain-specific pre-trained language models (PLMs) have achieved state-of-the-art results. However, it is under-investigated to design a unified paradigm to inject domain knowledge in the PLM fine-tuning stage. We propose KnowledgeDA, a unified domain language model development service to enhance the task-specific training procedure with domain knowledge graphs. Given domain-specific task texts input, KnowledgeDA can automatically generate a domain-specific language model following three steps: (i) localize domain knowledge entities in texts via an embedding-similarity approach; (ii) generate augmented samples by retrieving replaceable domain entity pairs from two views of both knowledge graph and training data; (iii) select high-quality augmented samples for fine-tuning via confidence-based assessment. We implement a prototype of KnowledgeDA to learn language models for two domains, healthcare and software development. Experiments on domain-specific text classification and QA tasks verify the effectiveness and generalizability of KnowledgeDA.

Related papers

• Learning to Generalize Unseen Domains via Multi-Source Meta Learning for Text Classification [71.08024880298613]
  We study the multi-source Domain Generalization of text classification. We propose a framework to use multiple seen domains to train a model that can achieve high accuracy in an unseen domain.
  arXiv  Detail & Related papers  (2024-09-20T07:46:21Z)
• DG-PIC: Domain Generalized Point-In-Context Learning for Point Cloud Understanding [41.49771026674969]
  We introduce a novel, practical, multi-domain multi-task setting, handling multiple domains and multiple tasks within one unified model for domain generalized point cloud understanding. Our DG-PIC does not require any model updates during the testing and can handle unseen domains and multiple tasks, textiti.e., point cloud reconstruction, denoising, and registration, within one unified model.
  arXiv  Detail & Related papers  (2024-07-11T18:21:40Z)
• A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation [52.0964459842176]
  Current state-of-the-art dialogue systems heavily rely on extensive training datasets. We propose a novel data textbfAugmentation framework for textbfMulti-textbfDomain textbfDialogue textbfGeneration, referred to as textbfAMD$2$G. The AMD$2$G framework consists of a data augmentation process and a two-stage training approach: domain-agnostic training and domain adaptation training.
  arXiv  Detail & Related papers  (2024-06-14T09:52:27Z)
• Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis [33.86086075084374]
  Aspect-based sentiment analysis (ABSA) is an important subtask of sentiment analysis. We propose a Large Language Model-based Continual Learning (textttLLM-CL) model for ABSA.
  arXiv  Detail & Related papers  (2024-05-09T02:00:07Z)
• Adapting to Distribution Shift by Visual Domain Prompt Generation [34.19066857066073]
  We adapt a model at test-time using a few unlabeled data to address distribution shifts. We build a knowledge bank to learn the transferable knowledge from source domains. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.
  arXiv  Detail & Related papers  (2024-05-05T02:44:04Z)
• TAL: Two-stream Adaptive Learning for Generalizable Person Re-identification [115.31432027711202]
  We argue that both domain-specific and domain-invariant features are crucial for improving the generalization ability of re-id models. We name two-stream adaptive learning (TAL) to simultaneously model these two kinds of information. Our framework can be applied to both single-source and multi-source domain generalization tasks.
  arXiv  Detail & Related papers  (2021-11-29T01:27:42Z)
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources [68.31273535702256]
  We propose a novel instance-level MDA framework, named curriculum cycle-consistent generative adversarial network (C-CycleGAN) C-CycleGAN consists of three components: (1) pre-trained text encoder which encodes textual input from different domains into a continuous representation space, (2) intermediate domain generator with curriculum instance-level adaptation which bridges the gap across source and target domains, and (3) task classifier trained on the intermediate domain for final sentiment classification. We conduct extensive experiments on three benchmark datasets and achieve substantial gains over state-of-the-art DA approaches.
  arXiv  Detail & Related papers  (2020-11-17T14:50:55Z)
• DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis [71.40586258509394]
  We propose DomBERT, an extension of BERT to learn from both in-domain corpus and relevant domain corpora. Experiments are conducted on an assortment of tasks in aspect-based sentiment analysis, demonstrating promising results.
  arXiv  Detail & Related papers  (2020-04-28T21:07:32Z)
• Domain Adaption for Knowledge Tracing [65.86619804954283]
  We propose a novel adaptable framework, namely knowledge tracing (AKT) to address the DAKT problem. For the first aspect, we incorporate the educational characteristics (e.g., slip, guess, question texts) based on the deep knowledge tracing (DKT) to obtain a good performed knowledge tracing model. For the second aspect, we propose and adopt three domain adaptation processes. First, we pre-train an auto-encoder to select useful source instances for target model training.
  arXiv  Detail & Related papers  (2020-01-14T15:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.