An Embedding-based Joint Sentiment-Topic Model for Short Texts

• URL: http://arxiv.org/abs/2103.14410v1
• Date: Fri, 26 Mar 2021 11:41:21 GMT
• Title: An Embedding-based Joint Sentiment-Topic Model for Short Texts
• Abstract summary: We develop ELJST, an embedding enhanced generative joint sentiment-topic model. It can discover more coherent and diverse topics from short texts. Our results show an average improvement of 10% in topic coherence and 5% in topic diversification over baselines.
• Score: 15.241185591218265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Short text is a popular avenue of sharing feedback, opinions and reviews on social media, e-commerce platforms, etc. Many companies need to extract meaningful information (which may include thematic content as well as semantic polarity) out of such short texts to understand users' behaviour. However, obtaining high quality sentiment-associated and human interpretable themes still remains a challenge for short texts. In this paper we develop ELJST, an embedding enhanced generative joint sentiment-topic model that can discover more coherent and diverse topics from short texts. It uses Markov Random Field Regularizer that can be seen as a generalisation of skip-gram based models. Further, it can leverage higher-order semantic information appearing in word embedding, such as self-attention weights in graphical models. Our results show an average improvement of 10% in topic coherence and 5% in topic diversification over baselines. Finally, ELJST helps understand users' behaviour at more granular levels which can be explained. All these can bring significant values to the service and healthcare industries often dealing with customers.

Related papers

• Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts [7.927346467214517]
  We extend a pretrained language model with a layer that learns annotator-specific features.<n>Our model improves predictions of individual annotations and yields representations that reveal meaningful insights into annotators' moral perspectives.
  arXiv  Detail & Related papers  (2026-06-22T02:19:23Z)
• TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction [0.0]
  We present TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction.<n>We evaluate our approach on four diverse datasets, 20News, AgNewsTitle, Reddit, and TweetTopic.<n>Our method produces more interpretable topics, highlighting its potential for applications in social media data and web content analytics.
  arXiv  Detail & Related papers  (2025-12-07T07:01:28Z)
• The Medium Is Not the Message: Deconfounding Text Embeddings via Linear Concept Erasure [91.01653854955286]
  Embedding-based similarity metrics can be influenced by spurious attributes like the text's source or language.<n>This paper shows that a debiasing algorithm that removes information about observed confounders from the encoder representations substantially reduces these biases at a minimal computational cost.
  arXiv  Detail & Related papers  (2025-07-01T23:17:12Z)
• ASCIIEval: Benchmarking Models' Visual Perception in Text Strings via ASCII Art [83.95594027644124]
  We frame the problem as a recognition task, and construct a novel benchmark, ASCIIEval.<n>It covers over 3K samples with an elaborate categorization tree, along with a training set for further enhancement.<n>Given textual input, language models shows their visual perception ability on ASCII art concepts.<n>For image inputs, we reveal that open-source MLLMs suffer from a trade-off between fine-grained text recognition and collective visual perception.
  arXiv  Detail & Related papers  (2024-10-02T16:46:01Z)
• Hierarchical Knowledge Distillation on Text Graph for Data-limited Attribute Inference [5.618638372635474]
  We develop a text-graph-based few-shot learning model for attribute inferences on social media text data. Our model first constructs and refines a text graph using manifold learning and message passing. To further use cross-domain texts and unlabeled texts to improve few-shot performance, a hierarchical knowledge distillation is devised over text graph.
  arXiv  Detail & Related papers  (2024-01-10T05:50:34Z)
• Context Unlocks Emotions: Text-based Emotion Classification Dataset Auditing with Large Language Models [23.670143829183104]
  The lack of contextual information in text data can make the annotation process of text-based emotion classification datasets challenging. We propose a formal definition of textual context to motivate a prompting strategy to enhance such contextual information. Our method improves alignment between inputs and their human-annotated labels from both an empirical and human-evaluated standpoint.
  arXiv  Detail & Related papers  (2023-11-06T21:34:49Z)
• TopicGPT: A Prompt-based Topic Modeling Framework [77.72072691307811]
  We introduce TopicGPT, a prompt-based framework that uses large language models to uncover latent topics in a text collection. It produces topics that align better with human categorizations compared to competing methods. Its topics are also interpretable, dispensing with ambiguous bags of words in favor of topics with natural language labels and associated free-form descriptions.
  arXiv  Detail & Related papers  (2023-11-02T17:57:10Z)
• Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models [3.656114607436271]
  Existing text scoring methods require a large corpus, struggle with short texts, or require hand-labeled data. We develop a text scoring framework that leverages generative large language models (LLMs) We apply this approach to better understand speech reflecting aversion to specific political parties on Twitter.
  arXiv  Detail & Related papers  (2023-10-18T15:34:37Z)
• Enriching language models with graph-based context information to better understand textual data [0.15469452301122172]
  We experimentally demonstrate that graph-based contextualization into BERT model enhances its performance on an example of a classification task. Specifically, on Pubmed dataset, we observed a reduction in error from 8.51% to 7.96%, while increasing the number of parameters just by 1.6%.
  arXiv  Detail & Related papers  (2023-05-10T10:57:21Z)
• Summarization with Graphical Elements [55.5913491389047]
  We propose a new task: summarization with graphical elements. We collect a high quality human labeled dataset to support research into the task.
  arXiv  Detail & Related papers  (2022-04-15T17:16:41Z)
• Author Clustering and Topic Estimation for Short Texts [69.54017251622211]
  We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document. We also simultaneously cluster users, removing the need for post-hoc cluster estimation. Our method performs as well as -- or better -- than traditional approaches to problems arising in short text.
  arXiv  Detail & Related papers  (2021-06-15T20:55:55Z)
• Sentiment analysis in tweets: an assessment study from classical to modern text representation models [59.107260266206445]
  Short texts published on Twitter have earned significant attention as a rich source of information. Their inherent characteristics, such as the informal, and noisy linguistic style, remain challenging to many natural language processing (NLP) tasks. This study fulfils an assessment of existing language models in distinguishing the sentiment expressed in tweets by using a rich collection of 22 datasets.
  arXiv  Detail & Related papers  (2021-05-29T21:05:28Z)
• Hierarchical Text Interaction for Rating Prediction [8.400688907233398]
  We propose a novel Hierarchical Text Interaction model for rating prediction. We exploit semantic correlations between each user-item pair at different hierarchies. Experiments on five real-world datasets demonstrate that HTI outperforms state-of-the-art models by a large margin.
  arXiv  Detail & Related papers  (2020-10-15T09:52:40Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
  We present the first application of the BERTSum model to conversational language. We generate abstractive summaries of narrated instructional videos across a wide variety of topics. We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
  arXiv  Detail & Related papers  (2020-08-21T20:59:34Z)
• Topic Modeling on User Stories using Word Mover's Distance [4.378337862197529]
  This paper focuses on topic modeling as a means to identify topics within a large set of crowd-generated user stories. We evaluate the approaches on a publicly available set of 2,966 user stories written and categorized by crowd workers.
  arXiv  Detail & Related papers  (2020-07-10T11:05:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.