Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based
Sentiment Analysis
- URL: http://arxiv.org/abs/2009.07964v4
- Date: Wed, 28 Oct 2020 08:19:36 GMT
- Title: Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based
Sentiment Analysis
- Authors: Xiaoyu Xing, Zhijing Jin, Di Jin, Bingning Wang, Qi Zhang, and
Xuanjing Huang
- Abstract summary: Aspect-based sentiment analysis (ABSA) aims to predict the sentiment towards a specific aspect in the text.
Existing ABSA test sets cannot be used to probe whether a model can distinguish the sentiment of the target aspect from the non-target aspects.
We generate new examples to disentangle the confounding sentiments of the non-target aspects from the target aspect's sentiment.
- Score: 71.40390724765903
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Aspect-based sentiment analysis (ABSA) aims to predict the sentiment towards
a specific aspect in the text. However, existing ABSA test sets cannot be used
to probe whether a model can distinguish the sentiment of the target aspect
from the non-target aspects. To solve this problem, we develop a simple but
effective approach to enrich ABSA test sets. Specifically, we generate new
examples to disentangle the confounding sentiments of the non-target aspects
from the target aspect's sentiment. Based on the SemEval 2014 dataset, we
construct the Aspect Robustness Test Set (ARTS) as a comprehensive probe of the
aspect robustness of ABSA models. Over 92% data of ARTS show high fluency and
desired sentiment on all aspects by human evaluation. Using ARTS, we analyze
the robustness of nine ABSA models, and observe, surprisingly, that their
accuracy drops by up to 69.73%. We explore several ways to improve aspect
robustness, and find that adversarial training can improve models' performance
on ARTS by up to 32.85%. Our code and new test set are available at
https://github.com/zhijing-jin/ARTS_TestSet
Related papers
- Deep Content Understanding Toward Entity and Aspect Target Sentiment Analysis on Foundation Models [0.8602553195689513]
Entity-Aspect Sentiment Triplet Extraction (EASTE) is a novel Aspect-Based Sentiment Analysis task.
Our research aims to achieve high performance on the EASTE task and investigates the impact of model size, type, and adaptation techniques on task performance.
Ultimately, we provide detailed insights and achieving state-of-the-art results in complex sentiment analysis.
arXiv Detail & Related papers (2024-07-04T16:48:14Z) - Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for
Advanced Object Detection [55.2480439325792]
We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head.
The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement.
This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
arXiv Detail & Related papers (2023-11-21T19:49:13Z) - A Weak Supervision Approach for Few-Shot Aspect Based Sentiment [39.33888584498155]
Weak supervision on abundant unlabeled data can be leveraged to improve few-shot performance in sentiment analysis tasks.
We propose a pipeline approach to construct a noisy ABSA dataset, and we use it to adapt a pre-trained sequence-to-sequence model to the ABSA tasks.
Our proposed method preserves the full fine-tuning performance while showing significant improvements (15.84% absolute F1) in the few-shot learning scenario.
arXiv Detail & Related papers (2023-05-19T19:53:54Z) - On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model,
Data, and Training [109.9218185711916]
Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind social media texts or reviews.
We propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training.
arXiv Detail & Related papers (2023-04-19T11:07:43Z) - Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with
DeBERTa [23.00810941211685]
Aspect-Based Sentiment Analysis (ABSA) is a fine-grained task in the field of sentiment analysis.
Recent DeBERTa model (Decoding-enhanced BERT with disentangled attention) to solve Aspect-Based Sentiment Analysis problem.
arXiv Detail & Related papers (2022-07-06T03:50:31Z) - A Simple Information-Based Approach to Unsupervised Domain-Adaptive
Aspect-Based Sentiment Analysis [58.124424775536326]
We propose a simple but effective technique based on mutual information to extract their term.
Experiment results show that our proposed method outperforms the state-of-the-art methods for cross-domain ABSA by 4.32% Micro-F1.
arXiv Detail & Related papers (2022-01-29T10:18:07Z) - Interventional Aspect-Based Sentiment Analysis [10.974711813144554]
We propose a simple yet effective method, namely, Sentiment Adjustment (SENTA), by applying a backdoor adjustment to disentangle those confounding factors.
Experimental results on the Aspect Robustness Test Set (ARTS) dataset demonstrate that our approach improves the performance while maintaining accuracy in the original test set.
arXiv Detail & Related papers (2021-04-20T07:54:29Z) - Understanding Pre-trained BERT for Aspect-based Sentiment Analysis [71.40586258509394]
This paper analyzes the pre-trained hidden representations learned from reviews on BERT for tasks in aspect-based sentiment analysis (ABSA)
It is not clear how the general proxy task of (masked) language model trained on unlabeled corpus without annotations of aspects or opinions can provide important features for downstream tasks in ABSA.
arXiv Detail & Related papers (2020-10-31T02:21:43Z) - Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional
Networks and Syntax-based Regulation [89.38054401427173]
Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect.
dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA.
We propose a novel graph-based deep learning model to overcome these two issues.
arXiv Detail & Related papers (2020-10-26T07:36:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.