An Interpretable and Uncertainty Aware Multi-Task Framework for
Multi-Aspect Sentiment Analysis
- URL: http://arxiv.org/abs/2009.09112v2
- Date: Mon, 31 May 2021 03:44:49 GMT
- Title: An Interpretable and Uncertainty Aware Multi-Task Framework for
Multi-Aspect Sentiment Analysis
- Authors: Tian Shi and Ping Wang and Chandan K. Reddy
- Abstract summary: Document-level Multi-aspect Sentiment Classification (DMSC) is a challenging and imminent problem.
We propose a deliberate self-attention-based deep neural network model, namely FEDAR, for the DMSC problem.
FEDAR can achieve competitive performance while also being able to interpret the predictions made.
- Score: 15.755185152760083
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, several online platforms have seen a rapid increase in the
number of review systems that request users to provide aspect-level feedback.
Document-level Multi-aspect Sentiment Classification (DMSC), where the goal is
to predict the ratings/sentiment from a review at an individual aspect level,
has become a challenging and imminent problem. To tackle this challenge, we
propose a deliberate self-attention-based deep neural network model, namely
FEDAR, for the DMSC problem, which can achieve competitive performance while
also being able to interpret the predictions made. FEDAR is equipped with a
highway word embedding layer to transfer knowledge from pre-trained word
embeddings, an RNN encoder layer with output features enriched by pooling and
factorization techniques, and a deliberate self-attention layer. In addition,
we also propose an Attention-driven Keywords Ranking (AKR) method, which can
automatically discover aspect keywords and aspect-level opinion keywords from
the review corpus based on the attention weights. These keywords are
significant for rating predictions by FEDAR. Since crowdsourcing annotation can
be an alternate way to recover missing ratings of reviews, we propose a
LEcture-AuDience (LEAD) strategy to estimate model uncertainty in the context
of multi-task learning, so that valuable human resources can focus on the most
uncertain predictions. Our extensive set of experiments on five different
open-domain DMSC datasets demonstrate the superiority of the proposed FEDAR and
LEAD models. We further introduce two new DMSC datasets in the healthcare
domain and benchmark different baseline models and our models on them.
Attention weights visualization results and visualization of aspect and opinion
keywords demonstrate the interpretability of our model and the effectiveness of
our AKR method.
Related papers
- Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness.
Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings.
This paper introduces an innovative multi-modal prompt-based methodology for IQA.
arXiv Detail & Related papers (2024-04-23T11:45:32Z) - VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models [57.43276586087863]
Large Vision-Language Models (LVLMs) suffer from hallucination issues, wherein the models generate plausible-sounding but factually incorrect outputs.
Existing benchmarks are often limited in scope, focusing mainly on object hallucinations.
We introduce a multi-dimensional benchmark covering objects, attributes, and relations, with challenging images selected based on associative biases.
arXiv Detail & Related papers (2024-04-22T04:49:22Z) - Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained
Ship Classification [62.425462136772666]
Fine-grained ship classification in remote sensing (RS-FGSC) poses a significant challenge due to the high similarity between classes and the limited availability of labeled data.
Recent advancements in large pre-trained Vision-Language Models (VLMs) have demonstrated impressive capabilities in few-shot or zero-shot learning.
This study delves into harnessing the potential of VLMs to enhance classification accuracy for unseen ship categories.
arXiv Detail & Related papers (2024-03-13T05:48:58Z) - Debiasing Multimodal Large Language Models [61.6896704217147]
Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing.
Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image.
To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
arXiv Detail & Related papers (2024-03-08T12:35:07Z) - Overcoming Pitfalls in Graph Contrastive Learning Evaluation: Toward
Comprehensive Benchmarks [60.82579717007963]
We introduce an enhanced evaluation framework designed to more accurately gauge the effectiveness, consistency, and overall capability of Graph Contrastive Learning (GCL) methods.
arXiv Detail & Related papers (2024-02-24T01:47:56Z) - HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation [20.178644251662316]
We introduce the hierarchical graph of thoughts (HGOT) to enhance the retrieval of pertinent passages during in-context learning.
The framework employs the divide-and-conquer strategy to break down complex queries into manageable sub-queries.
It refines self-consistency majority voting for answer selection, which incorporates the recently proposed citation recall and precision metrics.
arXiv Detail & Related papers (2024-02-14T18:41:19Z) - ScatterUQ: Interactive Uncertainty Visualizations for Multiclass Deep Learning Problems [0.0]
ScatterUQ is an interactive system that provides targeted visualizations to allow users to better understand model performance in context-driven uncertainty settings.
We demonstrate the effectiveness of ScatterUQ to explain model uncertainty for a multiclass image classification on a distance-aware neural network trained on Fashion-MNIST.
Our results indicate that the ScatterUQ system should scale to arbitrary, multiclass datasets.
arXiv Detail & Related papers (2023-08-08T21:17:03Z) - Uncertainty Estimation by Fisher Information-based Evidential Deep
Learning [61.94125052118442]
Uncertainty estimation is a key factor that makes deep learning reliable in practical applications.
We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL)
In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
arXiv Detail & Related papers (2023-03-03T16:12:59Z) - Don't Be So Sure! Boosting ASR Decoding via Confidence Relaxation [7.056222499095849]
beam search seeks the transcript with the greatest likelihood computed using the predicted distribution.
We show that recently proposed Self-Supervised Learning (SSL)-based ASR models tend to yield exceptionally confident predictions.
We propose a decoding procedure that improves the performance of fine-tuned ASR models.
arXiv Detail & Related papers (2022-12-27T06:42:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.