Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs

• URL: http://arxiv.org/abs/2210.10144v1
• Date: Tue, 18 Oct 2022 20:18:42 GMT
• Title: Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs
• Authors: Phillip Howard, Arden Ma, Vasudev Lal, Ana Paula Simoes, Daniel Korat, Oren Pereg, Moshe Wasserblat, Gadi Singer
• Abstract summary: We propose a novel approach for automatically constructing domain-specific knowledge graphs that contain information relevant to the identification of aspect terms. We demonstrate state-of-the-art performance on benchmark datasets for cross-domain aspect term extraction using our approach and investigate how the amount of external knowledge available to the Transformer impacts model performance.
• Score: 3.662157175955389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The extraction of aspect terms is a critical step in fine-grained sentiment analysis of text. Existing approaches for this task have yielded impressive results when the training and testing data are from the same domain. However, these methods show a drastic decrease in performance when applied to cross-domain settings where the domain of the testing data differs from that of the training data. To address this lack of extensibility and robustness, we propose a novel approach for automatically constructing domain-specific knowledge graphs that contain information relevant to the identification of aspect terms. We introduce a methodology for injecting information from these knowledge graphs into Transformer models, including two alternative mechanisms for knowledge insertion: via query enrichment and via manipulation of attention patterns. We demonstrate state-of-the-art performance on benchmark datasets for cross-domain aspect term extraction using our approach and investigate how the amount of external knowledge available to the Transformer impacts model performance.

Related papers

• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture. We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Selective Domain-Invariant Feature for Generalizable Deepfake Detection [21.671221284842847]
  We propose a novel framework which reduces the sensitivity to face forgery by fusing content features and styles. Both qualitative and quantitative results in existing benchmarks and proposals demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-03-19T13:09:19Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Mere Contrastive Learning for Cross-Domain Sentiment Analysis [23.350121129347556]
  Cross-domain sentiment analysis aims to predict the sentiment of texts in the target domain using the model trained on the source domain. Previous studies are mostly cross-entropy-based methods for the task, which suffer from instability and poor generalization. We propose a modified contrastive objective with in-batch negative samples so that the sentence representations from the same class will be pushed close.
  arXiv  Detail & Related papers  (2022-08-18T07:25:55Z)
• Disentanglement by Cyclic Reconstruction [0.0]
  In supervised learning, information specific to the dataset used for training, but irrelevant to the task at hand, may remain encoded in the extracted representations. We propose splitting the information into a task-related representation and its complementary context representation. We then adapt this method to the unsupervised domain adaptation problem, consisting of training a model capable of performing on both a source and a target domain.
  arXiv  Detail & Related papers  (2021-12-24T07:47:59Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• Inserting Information Bottlenecks for Attribution in Transformers [46.77580577396633]
  We apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers.
  arXiv  Detail & Related papers  (2020-12-27T00:35:43Z)
• Towards Domain-Agnostic Contrastive Learning [103.40783553846751]
  We propose a novel domain-agnostic approach to contrastive learning, named DACL. Key to our approach is the use of Mixup noise to create similar and dissimilar examples by mixing data samples differently either at the input or hidden-state levels. Our results show that DACL not only outperforms other domain-agnostic noising methods, such as Gaussian-noise, but also combines well with domain-specific methods, such as SimCLR.
  arXiv  Detail & Related papers  (2020-11-09T13:41:56Z)
• Self-Supervised Domain Adaptation with Consistency Training [0.2462953128215087]
  We consider the problem of unsupervised domain adaptation for image classification. We create a self-supervised pretext task by augmenting the unlabeled data with a certain type of transformation. We force the representation of the augmented data to be consistent with that of the original data.
  arXiv  Detail & Related papers  (2020-10-15T06:03:47Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.