Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis

• URL: http://arxiv.org/abs/2007.01189v3
• Date: Sun, 19 Jul 2020 08:50:19 GMT
• Title: Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis
• Authors: Avinash Madasu and Vijjini Anvesh Rao
• Abstract summary: We propose a model-independent framework - Sequential Domain Adaptation (SDA) Our experiments show that the proposed framework enables simple architectures such as CNNs to outperform complex state-of-the-art models in domain adaptation of sentiment analysis (SA) In addition, we observe that the effectiveness of a harder first Anti-Curriculum ordering of source domains leads to maximum performance.
• Score: 3.1473798197405944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Elastic Weight Consolidation (EWC) is a technique used in overcoming catastrophic forgetting between successive tasks trained on a neural network. We use this phenomenon of information sharing between tasks for domain adaptation. Training data for tasks such as sentiment analysis (SA) may not be fairly represented across multiple domains. Domain Adaptation (DA) aims to build algorithms that leverage information from source domains to facilitate performance on an unseen target domain. We propose a model-independent framework - Sequential Domain Adaptation (SDA). SDA draws on EWC for training on successive source domains to move towards a general domain solution, thereby solving the problem of domain adaptation. We test SDA on convolutional, recurrent, and attention-based architectures. Our experiments show that the proposed framework enables simple architectures such as CNNs to outperform complex state-of-the-art models in domain adaptation of SA. In addition, we observe that the effectiveness of a harder first Anti-Curriculum ordering of source domains leads to maximum performance.

Related papers

• Domain Adaptation from Scratch [24.612696638386623]
  We present a new learning setup, domain adaptation from scratch'', which we believe to be crucial for extending the reach of NLP to sensitive domains. In this setup, we aim to efficiently annotate data from a set of source domains such that the trained model performs well on a sensitive target domain. Our study compares several approaches for this challenging setup, ranging from data selection and domain adaptation algorithms to active learning paradigms.
  arXiv  Detail & Related papers  (2022-09-02T05:55:09Z)
• Domain Adaptation Principal Component Analysis: base linear method for learning with out-of-distribution data [55.41644538483948]
  Domain adaptation is a popular paradigm in modern machine learning. We present a method called Domain Adaptation Principal Component Analysis (DAPCA) DAPCA finds a linear reduced data representation useful for solving the domain adaptation task.
  arXiv  Detail & Related papers  (2022-08-28T21:10:56Z)
• Feed-Forward Latent Domain Adaptation [17.71179872529747]
  We study a new highly-practical problem setting that enables resource-constrained edge devices to adapt a pre-trained model to their local data distributions. Considering limitations of edge devices, we aim to only use a pre-trained model and adapt it in a feed-forward way, without using back-propagation and without access to the source data. Our solution is to meta-learn a network capable of embedding the mixed-relevance target dataset and dynamically adapting inference for target examples using cross-attention.
  arXiv  Detail & Related papers  (2022-07-15T17:37:42Z)
• Domain-Agnostic Prior for Transfer Semantic Segmentation [197.9378107222422]
  Unsupervised domain adaptation (UDA) is an important topic in the computer vision community. We present a mechanism that regularizes cross-domain representation learning with a domain-agnostic prior (DAP) Our research reveals that UDA benefits much from better proxies, possibly from other data modalities.
  arXiv  Detail & Related papers  (2022-04-06T09:13:25Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources [68.31273535702256]
  We propose a novel instance-level MDA framework, named curriculum cycle-consistent generative adversarial network (C-CycleGAN) C-CycleGAN consists of three components: (1) pre-trained text encoder which encodes textual input from different domains into a continuous representation space, (2) intermediate domain generator with curriculum instance-level adaptation which bridges the gap across source and target domains, and (3) task classifier trained on the intermediate domain for final sentiment classification. We conduct extensive experiments on three benchmark datasets and achieve substantial gains over state-of-the-art DA approaches.
  arXiv  Detail & Related papers  (2020-11-17T14:50:55Z)
• Domain Conditioned Adaptation Network [90.63261870610211]
  We propose a Domain Conditioned Adaptation Network (DCAN) to excite distinct convolutional channels with a domain conditioned channel attention mechanism. This is the first work to explore the domain-wise convolutional channel activation for deep DA networks.
  arXiv  Detail & Related papers  (2020-05-14T04:23:24Z)
• Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
  Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them. We propose a generic framework based on graph embedding. We show that the proposed approach leads to a powerful Domain Adaptation framework.
  arXiv  Detail & Related papers  (2020-03-09T12:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.