DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis

• URL: http://arxiv.org/abs/2004.13816v1
• Date: Tue, 28 Apr 2020 21:07:32 GMT
• Title: DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
• Authors: Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
• Abstract summary: We propose DomBERT, an extension of BERT to learn from both in-domain corpus and relevant domain corpora. Experiments are conducted on an assortment of tasks in aspect-based sentiment analysis, demonstrating promising results.
• Score: 71.40586258509394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper focuses on learning domain-oriented language models driven by end tasks, which aims to combine the worlds of both general-purpose language models (such as ELMo and BERT) and domain-specific language understanding. We propose DomBERT, an extension of BERT to learn from both in-domain corpus and relevant domain corpora. This helps in learning domain language models with low-resources. Experiments are conducted on an assortment of tasks in aspect-based sentiment analysis, demonstrating promising results.

Related papers

• Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis [33.86086075084374]
  Aspect-based sentiment analysis (ABSA) is an important subtask of sentiment analysis. We propose a Large Language Model-based Continual Learning (textttLLM-CL) model for ABSA.
  arXiv  Detail & Related papers  (2024-05-09T02:00:07Z)
• Large Language Models as Planning Domain Generators [21.61328475841776]
  Large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. LLMs exhibit a moderate level of proficiency in generating correct planning domains from natural language descriptions.
  arXiv  Detail & Related papers  (2024-04-02T19:39:23Z)
• Cross-domain Chinese Sentence Pattern Parsing [67.1381983012038]
  Sentence Pattern Structure (SPS) parsing is a syntactic analysis method primarily employed in language teaching. Existing SPSs rely heavily on textbook corpora for training, lacking cross-domain capability. This paper proposes an innovative approach leveraging large language models (LLMs) within a self-training framework.
  arXiv  Detail & Related papers  (2024-02-26T05:30:48Z)
• Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning [48.22913073217633]
  Large language models (LLMs) have showcased their capability with few-shot inference known as in-context learning. In this paper, we study the UDA problem under an in-context learning setting to adapt language models from the source domain to the target domain without any target labels. We devise different prompting and training strategies, accounting for different LM architectures to learn the target distribution via language modeling.
  arXiv  Detail & Related papers  (2023-11-20T06:06:20Z)
• Adapting Large Language Models to Domains via Reading Comprehension [86.24451681746676]
  We explore how continued pre-training on domain-specific corpora influences large language models. We show that training on the raw corpora endows the model with domain knowledge, but drastically hurts its ability for question answering. We propose a simple method for transforming raw corpora into reading comprehension texts.
  arXiv  Detail & Related papers  (2023-09-18T07:17:52Z)
• A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks [10.28161912127425]
  We propose KnowledgeDA, a unified domain language model development service to enhance the task-specific training procedure with domain knowledge graphs. We implement a prototype of KnowledgeDA to learn language models for two domains, healthcare and software development.
  arXiv  Detail & Related papers  (2022-12-10T09:18:43Z)
• Pretrained Domain-Specific Language Model for General Information Retrieval Tasks in the AEC Domain [5.949779668853556]
  It is unclear how domain corpora and domain-specific pretrained DL models can improve performance in various information retrieval tasks. This work explores the impacts of domain corpora and various transfer learning techniques on the performance of DL models for IR tasks. BERT-based models dramatically outperform traditional methods in all IR tasks, with maximum improvements of 5.4% and 10.1% in the F1 score.
  arXiv  Detail & Related papers  (2022-03-09T14:10:55Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)
• Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking [66.76141128555099]
  We propose a novel cross-lingual biomedical entity linking task (XL-BEL) We first investigate the ability of standard knowledge-agnostic as well as knowledge-enhanced monolingual and multilingual LMs beyond the standard monolingual English BEL task. We then address the challenge of transferring domain-specific knowledge in resource-rich languages to resource-poor ones.
  arXiv  Detail & Related papers  (2021-05-30T00:50:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.