CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling

• URL: http://arxiv.org/abs/2604.14489v2
• Date: Fri, 17 Apr 2026 15:19:39 GMT
• Title: CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling
• Abstract summary: We introduce CobwebTM, a lifelong hierarchical topic model based on incremental probabilistic concept formation.<n>By adapting the Cobweb algorithm to continuous document embeddings, CobwebTM constructs semantic hierarchies online.<n>Across diverse datasets, CobwebTM achieves strong topic coherence, stable topics over time, and high-quality hierarchies.
• Score: 3.3918676589553307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Topic modeling seeks to uncover latent semantic structure in text corpora with minimal supervision. Neural approaches achieve strong performance but require extensive tuning and struggle with lifelong learning due to catastrophic forgetting and fixed capacity, while classical probabilistic models lack flexibility and adaptability to streaming data. We introduce CobwebTM, a low-parameter lifelong hierarchical topic model based on incremental probabilistic concept formation. By adapting the Cobweb algorithm to continuous document embeddings, CobwebTM constructs semantic hierarchies online, enabling unsupervised topic discovery, dynamic topic creation, and hierarchical organization without predefining the number of topics. Across diverse datasets, CobwebTM achieves strong topic coherence, stable topics over time, and high-quality hierarchies, demonstrating that incremental symbolic concept formation combined with pretrained representations is an efficient approach to topic modeling.

Related papers

• Structured Neural Marked Point Processes for Interpretable Event Interaction Modeling [12.453735847515006]
  Multi-class event streams arise in numerous real-world applications, where uncovering structured, interpretable inter-event relationships, together with accurate prediction, remains a central challenge.<n>Existing neural point process models are highly expressive but encode event interactions in a black-box manner, preventing explicit discovery of structured dependencies.<n>We propose a structured neural marked point process (SNMPP) that achieves high modeling flexibility while enabling explicit event-wise and class-wise relationship discovery from data.
  arXiv  Detail & Related papers  (2026-05-17T17:56:22Z)
• The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents [13.516336193977144]
  We propose a sparse mixture-of-experts framework integrated into conventional diffusion models to solve text-to-image generation tasks.<n>Our approach iteratively refines visual tokens over multiple latent steps while efficiently sharing parameters via sparse selection of neural modules.<n> Comprehensive evaluation on class-conditioned ImageNet image generation tasks and additional studies on the GenEval and DPG benchmark demonstrate the superiority of the proposed method in enhancing model image generation performance.
  arXiv  Detail & Related papers  (2026-04-28T07:09:18Z)
• Multi-Scale Feature Fusion and Graph Neural Network Integration for Text Classification with Large Language Models [11.071281023081582]
  This study investigates a hybrid method for text classification that integrates deep feature extraction from large language models, multi-scale fusion through feature pyramids, and structured modeling with graph neural networks to enhance performance in complex semantic contexts.<n>The proposed method demonstrates significant advantages in robustness alignment experiments, outperforming existing models on ACC, F1-Score, AUC, and Precision, which verifies the effectiveness and stability of the framework.
  arXiv  Detail & Related papers  (2025-11-07T22:54:26Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models [3.4219049032524804]
  This paper proposes a modeling framework for dynamic topic evolution based on temporal large language models.<n>The proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text.
  arXiv  Detail & Related papers  (2025-10-12T13:50:41Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Discovering group dynamics in coordinated time series via hierarchical recurrent switching-state models [5.250223406627639]
  We seek a computationally efficient model for a collection of time series arising from multiple interacting entities (a.k.a. "agents")<n>Recent models of temporal patterns across individuals fail to incorporate explicit system-level collective behavior that can influence the trajectories of individual entities.<n>We employ a latent system-level discrete state Markov chain that provides top-down influence on latent entity-level chains which in turn govern the emission of each observed time series.
  arXiv  Detail & Related papers  (2024-01-26T16:06:01Z)
• On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling [29.277151061615434]
  We propose Transport Plan and Context-aware Hierarchical Topic Model (TraCo) TraCo constrains dependencies to ensure their sparsity and balance, and also regularizes topic hierarchy building with them. Rather than previously entangled decoding, it distributes different semantic granularity to topics at different levels by disentangled decoding.
  arXiv  Detail & Related papers  (2024-01-25T11:47:58Z)
• HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding [54.52651110749165]
  We present a novel framework that introduces hyperbolic embeddings to represent words and topics. With the tree-likeness property of hyperbolic space, the underlying semantic hierarchy can be better exploited to mine more interpretable topics.
  arXiv  Detail & Related papers  (2022-10-16T02:54:17Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• TopicNet: Semantic Graph-Guided Topic Discovery [51.71374479354178]
  Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner. We introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning.
  arXiv  Detail & Related papers  (2021-10-27T09:07:14Z)
• Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference [55.35176938713946]
  We develop deep autoencoding topic model (DATM) that uses a hierarchy of gamma distributions to construct its multi-stochastic-layer generative network. We propose a Weibull upward-downward variational encoder that deterministically propagates information upward via a deep neural network, followed by a downward generative model. The efficacy and scalability of our models are demonstrated on both unsupervised and supervised learning tasks on big corpora.
  arXiv  Detail & Related papers  (2020-06-15T22:22:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.