Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning

• URL: http://arxiv.org/abs/2103.03446v1
• Date: Fri, 5 Mar 2021 02:50:05 GMT
• Title: Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning
• Authors: Jinsong Su, Jialong Tang, Hui Jiang, Ziyao Lu, Yubin Ge, Linfeng Song, Deyi Xiong, Le Sun, Jiebo Luo
• Abstract summary: In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction. We propose a progressive self-supervised attention learning approach for attentional ABSA models. We integrate the proposed approach into three state-of-the-art neural ABSA models.
• Score: 103.0064298630794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction. However, such a mechanism suffers from one drawback: only a few frequent words with sentiment polarities are tended to be taken into consideration for final sentiment decision while abundant infrequent sentiment words are ignored by models. To deal with this issue, we propose a progressive self-supervised attention learning approach for attentional ABSA models. In this approach, we iteratively perform sentiment prediction on all training instances, and continually learn useful attention supervision information in the meantime. During training, at each iteration, context words with the highest impact on sentiment prediction, identified based on their attention weights or gradients, are extracted as words with active/misleading influence on the correct/incorrect prediction for each instance. Words extracted in this way are masked for subsequent iterations. To exploit these extracted words for refining ABSA models, we augment the conventional training objective with a regularization term that encourages ABSA models to not only take full advantage of the extracted active context words but also decrease the weights of those misleading words. We integrate the proposed approach into three state-of-the-art neural ABSA models. Experiment results and in-depth analyses show that our approach yields better attention results and significantly enhances the performance of all three models. We release the source code and trained models at https://github.com/DeepLearnXMU/PSSAttention.

Related papers

• Let's Rectify Step by Step: Improving Aspect-based Sentiment Analysis with Diffusion Models [36.482634643246264]
  We propose a novel diffusion model tailored for ABSA, which extracts the aspects progressively step by step. DiffusionABSA gradually adds noise to the aspect terms in the training process, subsequently learning a denoising process that progressively restores these terms in a reverse manner. To estimate the boundaries, we design a denoising neural network enhanced by a syntax-aware temporal attention mechanism.
  arXiv  Detail & Related papers  (2024-02-23T12:35:43Z)
• Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse [73.65112477688353]
  Recent studies have highlighted a phenomenon in large language models known as "the reversal curse" We contend that the reversal curse is partially a result of specific model training objectives. We propose a novel training method, BI Casual language modeling Optimization (BICO), designed to mitigate the reversal curse.
  arXiv  Detail & Related papers  (2023-11-13T17:01:12Z)
• What Do Deep Saliency Models Learn about Visual Attention? [28.023464783469738]
  We present a novel analytic framework that sheds light on the implicit features learned by saliency models. Our approach decomposes these implicit features into interpretable bases that are explicitly aligned with semantic attributes.
  arXiv  Detail & Related papers  (2023-10-14T23:15:57Z)
• Explaining Hate Speech Classification with Model Agnostic Methods [0.9990687944474738]
  The research goal of this paper is to bridge the gap between hate speech prediction and the explanations generated by the system to support its decision. This has been achieved by first predicting the classification of a text and then providing a posthoc, model agnostic and surrogate interpretability approach.
  arXiv  Detail & Related papers  (2023-05-30T19:52:56Z)
• Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis [67.41078214475341]
  We propose Dynamic Re-weighting BERT (DR-BERT) to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence. We then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA)
  arXiv  Detail & Related papers  (2022-03-30T14:48:46Z)
• Better Language Model with Hypernym Class Prediction [101.8517004687825]
  Class-based language models (LMs) have been long devised to address context sparsity in $n$-gram LMs. In this study, we revisit this approach in the context of neural LMs.
  arXiv  Detail & Related papers  (2022-03-21T01:16:44Z)
• SparseBERT: Rethinking the Importance Analysis in Self-attention [107.68072039537311]
  Transformer-based models are popular for natural language processing (NLP) tasks due to its powerful capacity. Attention map visualization of a pre-trained model is one direct method for understanding self-attention mechanism. We propose a Differentiable Attention Mask (DAM) algorithm, which can be also applied in guidance of SparseBERT design.
  arXiv  Detail & Related papers  (2021-02-25T14:13:44Z)
• Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation [89.38054401427173]
  Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect. dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA. We propose a novel graph-based deep learning model to overcome these two issues.
  arXiv  Detail & Related papers  (2020-10-26T07:36:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.