Semantic Component Analysis: Discovering Patterns in Short Texts Beyond Topics

• URL: http://arxiv.org/abs/2410.21054v1
• Date: Mon, 28 Oct 2024 14:09:52 GMT
• Title: Semantic Component Analysis: Discovering Patterns in Short Texts Beyond Topics
• Authors: Florian Eichin, Carolin Schuster, Georg Groh, Michael A. Hedderich,
• Abstract summary: We introduce Semantic Component Analysis (SCA), a novel topic modeling technique. We find multiple, nuanced semantic components beyond a single topic in short texts. Evaluated on multiple Twitter datasets, SCA matches the state-of-the-art method BERTopic in coherence and diversity.
• Score: 7.784473631566062
• License:
• Abstract: Topic modeling is a key method in text analysis, but existing approaches are limited by assuming one topic per document or fail to scale efficiently for large, noisy datasets of short texts. We introduce Semantic Component Analysis (SCA), a novel topic modeling technique that overcomes these limitations by discovering multiple, nuanced semantic components beyond a single topic in short texts which we accomplish by introducing a decomposition step to the clustering-based topic modeling framework. Evaluated on multiple Twitter datasets, SCA matches the state-of-the-art method BERTopic in coherence and diversity, while uncovering at least double the semantic components and maintaining a noise rate close to zero while staying scalable and effective across languages, including an underrepresented one.

Related papers

• Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEs [25.915607750636333]
  We propose a novel approach that leverages large language models (LLMs) to extend short texts into more detailed sequences before applying topic modeling. Our method significantly improves short-text topic modeling performance, as demonstrated by extensive experiments on real-world datasets with extreme data sparsity.
  arXiv  Detail & Related papers  (2024-10-04T01:28:56Z)
• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions [63.11097464396147]
  We introduce a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. We also introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-27T15:59:37Z)
• Prompting Large Language Models for Topic Modeling [10.31712610860913]
  We propose PromptTopic, a novel topic modeling approach that harnesses the advanced language understanding of large language models (LLMs) It involves extracting topics at the sentence level from individual documents, then aggregating and condensing these topics into a predefined quantity, ultimately providing coherent topics for texts of varying lengths. We benchmark PromptTopic against the state-of-the-art baselines on three vastly diverse datasets, establishing its proficiency in discovering meaningful topics.
  arXiv  Detail & Related papers  (2023-12-15T11:15:05Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Let the Pretrained Language Models "Imagine" for Short Texts Topic Modeling [29.87929724277381]
  In short texts, co-occurrence information is minimal, which results in feature sparsity in document representation. Existing topic models (probabilistic or neural) mostly fail to mine patterns from them to generate coherent topics. We extend short text into longer sequences using existing pre-trained language models (PLMs)
  arXiv  Detail & Related papers  (2023-10-24T00:23:30Z)
• Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations [35.74225306947918]
  We propose a joint latent space learning and clustering framework built upon PLM embeddings. Our model effectively leverages the strong representation power and superb linguistic features brought by PLMs for topic discovery.
  arXiv  Detail & Related papers  (2022-02-09T17:26:08Z)
• Author Clustering and Topic Estimation for Short Texts [69.54017251622211]
  We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document. We also simultaneously cluster users, removing the need for post-hoc cluster estimation. Our method performs as well as -- or better -- than traditional approaches to problems arising in short text.
  arXiv  Detail & Related papers  (2021-06-15T20:55:55Z)
• BATS: A Spectral Biclustering Approach to Single Document Topic Modeling and Segmentation [17.003488045214972]
  Existing topic modeling and text segmentation methodologies generally require large datasets for training, limiting their capabilities when only small collections of text are available. In developing a methodology to handle single documents, we face two major challenges. First is sparse information: with access to only one document, we cannot train traditional topic models or deep learning algorithms. Second is significant noise: a considerable portion of words in any single document will produce only noise and not help discern topics or segments.
  arXiv  Detail & Related papers  (2020-08-05T16:34:33Z)
• Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
  We propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
  arXiv  Detail & Related papers  (2020-07-13T11:03:09Z)
• How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context [59.13515950353125]
  We present a grammar-based decoding semantic parsing and adapt typical context modeling methods on top of it. We evaluate 13 context modeling methods on two large cross-domain datasets, and our best model achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-02-03T11:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.