A Survey on Neural Topic Models: Methods, Applications, and Challenges

• URL: http://arxiv.org/abs/2401.15351v2
• Date: Mon, 24 Jun 2024 12:08:06 GMT
• Title: A Survey on Neural Topic Models: Methods, Applications, and Challenges
• Authors: Xiaobao Wu, Thong Nguyen, Anh Tuan Luu,
• Abstract summary: Topic models have been prevalent for decades to discover latent topics and infer topic proportions of documents in an unsupervised fashion. The rise of neural networks has facilitated the emergence of a new research field -- Neural Topic Models (NTMs) In this paper, we present a comprehensive survey on neural topic models concerning methods, applications, and challenges.
• Score: 32.510888679613004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Topic models have been prevalent for decades to discover latent topics and infer topic proportions of documents in an unsupervised fashion. They have been widely used in various applications like text analysis and context recommendation. Recently, the rise of neural networks has facilitated the emergence of a new research field -- Neural Topic Models (NTMs). Different from conventional topic models, NTMs directly optimize parameters without requiring model-specific derivations. This endows NTMs with better scalability and flexibility, resulting in significant research attention and plentiful new methods and applications. In this paper, we present a comprehensive survey on neural topic models concerning methods, applications, and challenges. Specifically, we systematically organize current NTM methods according to their network structures and introduce the NTMs for various scenarios like short texts and bilingual documents. We also discuss a wide range of popular applications built on NTMs. Finally, we highlight the challenges confronted by NTMs to inspire future research. We accompany this survey with a repository for easier access to the mentioned paper resources: https://github.com/bobxwu/Paper-Neural-Topic-Models.

Related papers

• Historia Magistra Vitae: Dynamic Topic Modeling of Roman Literature using Neural Embeddings [10.095706051685665]
  We compare topic models built using traditional statistical models (LDA and NMF) and the BERT-based model. We find that while quantitative metrics prefer statistical models, qualitative evaluation finds better insights from the neural model.
  arXiv  Detail & Related papers  (2024-06-27T05:38:49Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• A Joint Learning Approach for Semi-supervised Neural Topic Modeling [25.104653662416023]
  We introduce the Label-Indexed Neural Topic Model (LI-NTM), which is the first effective upstream semi-supervised neural topic model. We find that LI-NTM outperforms existing neural topic models in document reconstruction benchmarks.
  arXiv  Detail & Related papers  (2022-04-07T04:42:17Z)
• Neural Topic Modeling with Deep Mutual Information Estimation [23.474848535821994]
  We propose a neural topic model which incorporates deep mutual information estimation. NTM-DMIE is a neural network method for topic learning. We evaluate NTM-DMIE on several metrics, including accuracy of text clustering, with topic representation, topic uniqueness and topic coherence.
  arXiv  Detail & Related papers  (2022-03-12T01:08:10Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Topic Modelling Meets Deep Neural Networks: A Survey [25.950652301810425]
  Topic modelling has been a successful technique for text analysis for almost twenty years. When topic modelling met deep neural networks, there emerged a new and increasingly popular research area, neural topic models. This paper provides a focused yet comprehensive overview of neural topic models for interested researchers in the AI community.
  arXiv  Detail & Related papers  (2021-02-28T12:59:28Z)
• Improving Neural Topic Models using Knowledge Distillation [84.66983329587073]
  We use knowledge distillation to combine the best attributes of probabilistic topic models and pretrained transformers. Our modular method can be straightforwardly applied with any neural topic model to improve topic quality.
  arXiv  Detail & Related papers  (2020-10-05T22:49:16Z)
• Neural Topic Model via Optimal Transport [24.15046280736009]
  We present a new neural topic model via the theory of optimal transport (OT) Specifically, we propose to learn the topic distribution of a document by directly minimising its OT distance to the document's word distributions. Our proposed model can be trained efficiently with a differentiable loss.
  arXiv  Detail & Related papers  (2020-08-12T06:37:09Z)
• A Comprehensive Study on Temporal Modeling for Online Action Detection [50.558313106389335]
  Online action detection (OAD) is a practical yet challenging task, which has attracted increasing attention in recent years. This paper aims to provide a comprehensive study on temporal modeling for OAD including four meta types of temporal modeling methods. We present several hybrid temporal modeling methods, which outperform the recent state-of-the-art methods with sizable margins on THUMOS-14 and TVSeries.
  arXiv  Detail & Related papers  (2020-01-21T13:12:58Z)
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study [81.11161697133095]
  We take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers.
  arXiv  Detail & Related papers  (2020-01-12T04:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.