Multivariate Gaussian Topic Modelling: A novel approach to discover topics with greater semantic coherence

• URL: http://arxiv.org/abs/2503.15036v2
• Date: Sat, 01 Nov 2025 12:16:39 GMT
• Title: Multivariate Gaussian Topic Modelling: A novel approach to discover topics with greater semantic coherence
• Authors: Satyajeet Sahoo, Jhareswar Maiti,
• Abstract summary: We propose a novel Multivariate Gaussian Topic Model (MGTM) to identify semantically coherent topics.<n>This approach is applied on 20 newsgroups dataset to demonstrate the interpretability benefits vis-a-vis 4 other benchmark models.<n>This model achieves a highest mean topic coherence (0.7) and median topic coherence (0.76) vis-a-vis the benchmark models, demonstrating high effectiveness in identifying interpretable, semantically coherent topics.
• Score: 3.6381860041528085
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: An important aspect of text mining involves information retrieval in form of discovery of semantic themes (topics) from documents using topic modelling. While generative topic models like Latent Dirichlet Allocation (LDA) or Latent Semantic Analysis (LSA) elegantly model topics as probability distributions and are useful in identifying latent topics from large document corpora with minimal supervision, they suffer from difficulty in topic interpretability and reduced performance in shorter texts. Here we propose a novel Multivariate Gaussian Topic Model (MGTM). In this approach topics are presented as Multivariate Gaussian Distributions and documents as Gaussian Mixture Models. Applying EM algorithm on a document corpus, the various constituent Multivariate Gaussian distributions corresponding to the latent topics and their respective parameters are identified. Analysis of the parameters of each distribution helps identify the respective topic keywords, and from these key-words topic annotations are carried out. This approach is applied on 20 newsgroups dataset to demonstrate the interpretability benefits vis-`a-vis 4 other benchmark models. The effectiveness of this model in capturing the semantic theme of the topics with high interpretability is examined by calculating the topic coherence and comparing the coherence values with benchmark models. This model achieves a highest mean topic coherence (0.7) and median topic coherence (0.76) vis-`a-vis the benchmark models, demonstrating high effectiveness in identifying interpretable, semantically coherent topics.

Related papers

• Measuring and Analyzing Intelligence via Contextual Uncertainty in Large Language Models using Information-Theoretic Metrics [0.0]
  We introduce a novel, task-agnostic approach to probe dynamics by creating a quantitative Cognitive Profile" for any given model.<n>Applying this methodology to several state-of-the-art LLMs across diverse texts, we uncover unique and consistent cognitive profiles that are sensitive to both model scale and text complexity.
  arXiv  Detail & Related papers  (2025-07-21T20:14:25Z)
• An Enhanced Model-based Approach for Short Text Clustering [58.60681789677676]
  Short text clustering has become increasingly important with the popularity of social media like Twitter, Google+, and Facebook.<n>Existing methods can be broadly categorized into two paradigms: topic model-based approaches and deep representation learning-based approaches.<n>We propose a collapsed Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture model (GSDMM), which effectively handles the sparsity and high dimensionality of short texts.<n>Based on several aspects of GSDMM that warrant further refinement, we propose an improved approach, GSDMM+, designed to further optimize its performance.
  arXiv  Detail & Related papers  (2025-07-18T10:07:42Z)
• Reliability of Topic Modeling [0.3759936323189418]
  We show that the standard practice for quantifying topic model reliability fails to capture essential aspects of the variation in two widely-used topic models. On synthetic and real-world data, we show that McDonald's $omega$ provides the best encapsulation of reliability.
  arXiv  Detail & Related papers  (2024-10-30T16:42:04Z)
• Investigating the Impact of Text Summarization on Topic Modeling [13.581341206178525]
  In this paper, an approach is proposed that further enhances topic modeling performance by utilizing a pre-trained large language model (LLM) Few shot prompting is used to generate summaries of different lengths to compare their impact on topic modeling. The proposed method yields better topic diversity and comparable coherence values compared to previous models.
  arXiv  Detail & Related papers  (2024-09-28T19:45:45Z)
• Explaining Datasets in Words: Statistical Models with Natural Language Parameters [66.69456696878842]
  We introduce a family of statistical models -- including clustering, time series, and classification models -- parameterized by natural language predicates.<n>We apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other.
  arXiv  Detail & Related papers  (2024-09-13T01:40:20Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• GINopic: Topic Modeling with Graph Isomorphism Network [0.8962460460173959]
  We introduce GINopic, a topic modeling framework based on graph isomorphism networks to capture the correlation between words.<n>We demonstrate the effectiveness of GINopic compared to existing topic models and highlight its potential for advancing topic modeling.
  arXiv  Detail & Related papers  (2024-04-02T17:18:48Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Topics in the Haystack: Extracting and Evaluating Topics beyond Coherence [0.0]
  We propose a method that incorporates a deeper understanding of both sentence and document themes. This allows our model to detect latent topics that may include uncommon words or neologisms. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task.
  arXiv  Detail & Related papers  (2023-03-30T12:24:25Z)
• A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling [0.0]
  This study presents the Latent Dirichlet Allocation (LDA) approach used to perform topic modelling. The visualisation provides an overarching view of the main topics while allowing and attributing deep meaning to the prevalence individual topic. The results suggest the terms ranked purely by considering their probability of the topic prevalence within the processed document.
  arXiv  Detail & Related papers  (2022-07-23T11:04:03Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• Topic Analysis for Text with Side Data [18.939336393665553]
  We introduce a hybrid generative probabilistic model that combines a neural network with a latent topic model. In the model, each document is modeled as a finite mixture over an underlying set of topics. Each topic is modeled as an infinite mixture over an underlying set of topic probabilities.
  arXiv  Detail & Related papers  (2022-03-01T22:06:30Z)
• Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations [35.74225306947918]
  We propose a joint latent space learning and clustering framework built upon PLM embeddings. Our model effectively leverages the strong representation power and superb linguistic features brought by PLMs for topic discovery.
  arXiv  Detail & Related papers  (2022-02-09T17:26:08Z)
• TopicNet: Semantic Graph-Guided Topic Discovery [51.71374479354178]
  Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner. We introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning.
  arXiv  Detail & Related papers  (2021-10-27T09:07:14Z)
• Author Clustering and Topic Estimation for Short Texts [69.54017251622211]
  We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document. We also simultaneously cluster users, removing the need for post-hoc cluster estimation. Our method performs as well as -- or better -- than traditional approaches to problems arising in short text.
  arXiv  Detail & Related papers  (2021-06-15T20:55:55Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)
• Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference [55.35176938713946]
  We develop deep autoencoding topic model (DATM) that uses a hierarchy of gamma distributions to construct its multi-stochastic-layer generative network. We propose a Weibull upward-downward variational encoder that deterministically propagates information upward via a deep neural network, followed by a downward generative model. The efficacy and scalability of our models are demonstrated on both unsupervised and supervised learning tasks on big corpora.
  arXiv  Detail & Related papers  (2020-06-15T22:22:56Z)
• Bayesian Sparse Factor Analysis with Kernelized Observations [67.60224656603823]
  Multi-view problems can be faced with latent variable models. High-dimensionality and non-linear issues are traditionally handled by kernel methods. We propose merging both approaches into single model.
  arXiv  Detail & Related papers  (2020-06-01T14:25:38Z)
• Keyword Assisted Topic Models [0.0]
  We show that providing a small number of keywords can substantially enhance the measurement performance of topic models. KeyATM provides more interpretable results, has better document classification performance, and is less sensitive to the number of topics than the standard topic models.
  arXiv  Detail & Related papers  (2020-04-13T14:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.