Unified Instance and Knowledge Alignment Pretraining for Aspect-based
Sentiment Analysis
- URL: http://arxiv.org/abs/2110.13398v3
- Date: Mon, 26 Jun 2023 04:24:50 GMT
- Title: Unified Instance and Knowledge Alignment Pretraining for Aspect-based
Sentiment Analysis
- Authors: Juhua Liu, Qihuang Zhong, Liang Ding, Hua Jin, Bo Du, Dacheng Tao
- Abstract summary: Aspect-based Sentiment Analysis (ABSA) aims to determine the sentiment polarity towards an aspect.
There always exists severe domain shift between the pretraining and downstream ABSA datasets.
We introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline.
- Score: 96.53859361560505
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Aspect-based Sentiment Analysis (ABSA) aims to determine the sentiment
polarity towards an aspect. Because of the expensive and limited labelled data,
the pretraining strategy has become the de-facto standard for ABSA. However,
there always exists severe domain shift between the pretraining and downstream
ABSA datasets, hindering the effective knowledge transfer when directly
finetuning and making the downstream task performs sub-optimal. To mitigate
such domain shift, we introduce a unified alignment pretraining framework into
the vanilla pretrain-finetune pipeline with both instance- and knowledge-level
alignments. Specifically, we first devise a novel coarse-to-fine retrieval
sampling approach to select target domain-related instances from the
large-scale pretraining dataset, thus aligning the instances between
pretraining and target domains (First Stage). Then, we introduce a knowledge
guidance-based strategy to further bridge the domain gap at the knowledge
level. In practice, we formulate the model pretrained on the sampled instances
into a knowledge guidance model and a learner model, respectively. On the
target dataset, we design an on-the-fly teacher-student joint fine-tuning
approach to progressively transfer the knowledge from the knowledge guidance
model to the learner model (Second Stage). Thereby, the learner model can
maintain more domain-invariant knowledge when learning new knowledge from the
target dataset. In the Third Stage, the learner model is finetuned to better
adapt its learned knowledge to the target dataset. Extensive experiments and
analyses on several ABSA benchmarks demonstrate the effectiveness and
universality of our proposed pretraining framework. Our source code and models
are publicly available at https://github.com/WHU-ZQH/UIKA.
Related papers
- Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance.
DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator.
Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
arXiv Detail & Related papers (2024-07-20T01:34:13Z) - Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training [23.56208527227504]
Source-free domain adaptation (SFDA) aims to adapt a source model trained on a fully-labeled source domain to a related but unlabeled target domain.
In the conventional SFDA pipeline, a large data (e.g. ImageNet) pre-trained feature extractor is used to initialize the source model.
We introduce an integrated framework to incorporate pre-trained networks into the target adaptation process.
arXiv Detail & Related papers (2024-05-05T14:48:13Z) - In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene
Classification [5.323049242720532]
Self-supervised learning has emerged as a promising approach for remote sensing image classification.
We present a study of different self-supervised pre-training strategies and evaluate their effect across 14 downstream datasets.
arXiv Detail & Related papers (2023-07-04T10:57:52Z) - Prior Knowledge Guided Unsupervised Domain Adaptation [82.9977759320565]
We propose a Knowledge-guided Unsupervised Domain Adaptation (KUDA) setting where prior knowledge about the target class distribution is available.
In particular, we consider two specific types of prior knowledge about the class distribution in the target domain: Unary Bound and Binary Relationship.
We propose a rectification module that uses such prior knowledge to refine model generated pseudo labels.
arXiv Detail & Related papers (2022-07-18T18:41:36Z) - A self-training framework for glaucoma grading in OCT B-scans [6.382852973055393]
We present a self-training-based framework for glaucoma grading using OCT B-scans under the presence of domain shift.
A two-step learning methodology resorts to pseudo-labels generated during the first step to augment the training dataset on the target domain.
We propose a novel glaucoma-specific backbone which introduces residual and attention modules via skip-connections to refine the embedding features of the latent space.
arXiv Detail & Related papers (2021-11-23T10:33:55Z) - Source-Free Open Compound Domain Adaptation in Semantic Segmentation [99.82890571842603]
In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model.
We propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level.
Our method produces state-of-the-art results on the C-Driving dataset.
arXiv Detail & Related papers (2021-06-07T08:38:41Z) - TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain
Gait Recognition [77.77786072373942]
This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition.
We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space.
Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
arXiv Detail & Related papers (2021-02-09T03:07:07Z) - Source Data-absent Unsupervised Domain Adaptation through Hypothesis
Transfer and Labeling Transfer [137.36099660616975]
Unsupervised adaptation adaptation (UDA) aims to transfer knowledge from a related but different well-labeled source domain to a new unlabeled target domain.
Most existing UDA methods require access to the source data, and thus are not applicable when the data are confidential and not shareable due to privacy concerns.
This paper aims to tackle a realistic setting with only a classification model available trained over, instead of accessing to the source data.
arXiv Detail & Related papers (2020-12-14T07:28:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.