vONTSS: vMF based semi-supervised neural topic modeling with optimal
transport
- URL: http://arxiv.org/abs/2307.01226v2
- Date: Sat, 16 Sep 2023 16:51:17 GMT
- Title: vONTSS: vMF based semi-supervised neural topic modeling with optimal
transport
- Authors: Weijie Xu, Xiaoyu Jiang, Srinivasan H. Sengamedu, Francis Iannacci,
Jinjin Zhao
- Abstract summary: This work presents a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport.
Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity.
It is also much faster than the state-of-the-art weakly supervised text classification method while achieving similar classification performance.
- Score: 6.874745415692134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, Neural Topic Models (NTM), inspired by variational autoencoders,
have attracted a lot of research interest; however, these methods have limited
applications in the real world due to the challenge of incorporating human
knowledge. This work presents a semi-supervised neural topic modeling method,
vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and
optimal transport. When a few keywords per topic are provided, vONTSS in the
semi-supervised setting generates potential topics and optimizes topic-keyword
quality and topic classification. Experiments show that vONTSS outperforms
existing semi-supervised topic modeling methods in classification accuracy and
diversity. vONTSS also supports unsupervised topic modeling. Quantitative and
qualitative experiments show that vONTSS in the unsupervised setting
outperforms recent NTMs on multiple aspects: vONTSS discovers highly clustered
and coherent topics on benchmark datasets. It is also much faster than the
state-of-the-art weakly supervised text classification method while achieving
similar classification performance. We further prove the equivalence of optimal
transport loss and cross-entropy loss at the global minimum.
Related papers
- Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learning [23.816433328623397]
Overemphasizing likelihood without incorporating topic regularization can lead to an overly expansive latent space for topic modeling.
We propose a novel NTM framework, named ContraTopic, that integrates a differentiable regularizer capable of evaluating multiple facets of topic interpretability.
Our approach consistently produces topics with superior interpretability compared to state-of-the-art NTMs.
arXiv Detail & Related papers (2024-12-23T07:07:06Z) - GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models.
GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies.
We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
arXiv Detail & Related papers (2024-06-01T08:01:05Z) - Improving the TENOR of Labeling: Re-evaluating Topic Models for Content
Analysis [5.757610495733924]
We conduct the first evaluation of neural, supervised and classical topic models in an interactive task based setting.
We show that current automated metrics do not provide a complete picture of topic modeling capabilities.
arXiv Detail & Related papers (2024-01-29T17:54:04Z) - Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual
Downstream Tasks [55.36987468073152]
This paper proposes a novel Dual-Guided Spatial-Channel-Temporal (DG-SCT) attention mechanism.
The DG-SCT module incorporates trainable cross-modal interaction layers into pre-trained audio-visual encoders.
Our proposed model achieves state-of-the-art results across multiple downstream tasks, including AVE, AVVP, AVS, and AVQA.
arXiv Detail & Related papers (2023-11-09T05:24:20Z) - Multi-View Clustering via Semi-non-negative Tensor Factorization [120.87318230985653]
We develop a novel multi-view clustering based on semi-non-negative tensor factorization (Semi-NTF)
Our model directly considers the between-view relationship and exploits the between-view complementary information.
In addition, we provide an optimization algorithm for the proposed method and prove mathematically that the algorithm always converges to the stationary KKT point.
arXiv Detail & Related papers (2023-03-29T14:54:19Z) - Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution
Detection [55.028065567756066]
Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications.
In this paper we propose an uncertainty quantification approach by modelling the distribution of features.
We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem.
We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
arXiv Detail & Related papers (2022-06-26T16:00:22Z) - A Joint Learning Approach for Semi-supervised Neural Topic Modeling [25.104653662416023]
We introduce the Label-Indexed Neural Topic Model (LI-NTM), which is the first effective upstream semi-supervised neural topic model.
We find that LI-NTM outperforms existing neural topic models in document reconstruction benchmarks.
arXiv Detail & Related papers (2022-04-07T04:42:17Z) - Revisiting LSTM Networks for Semi-Supervised Text Classification via
Mixed Objective Function [106.69643619725652]
We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results.
We report state-of-the-art results for text classification task on several benchmark datasets.
arXiv Detail & Related papers (2020-09-08T21:55:22Z) - Neural Topic Model via Optimal Transport [24.15046280736009]
We present a new neural topic model via the theory of optimal transport (OT)
Specifically, we propose to learn the topic distribution of a document by directly minimising its OT distance to the document's word distributions.
Our proposed model can be trained efficiently with a differentiable loss.
arXiv Detail & Related papers (2020-08-12T06:37:09Z) - Context Reinforced Neural Topic Modeling over Short Texts [15.487822291146689]
We propose a Context Reinforced Neural Topic Model (CRNTM)
CRNTM infers the topic for each word in a narrow range by assuming that each short text covers only a few salient topics.
Experiments on two benchmark datasets validate the effectiveness of the proposed model on both topic discovery and text classification.
arXiv Detail & Related papers (2020-08-11T06:41:53Z) - Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference [55.35176938713946]
We develop deep autoencoding topic model (DATM) that uses a hierarchy of gamma distributions to construct its multi-stochastic-layer generative network.
We propose a Weibull upward-downward variational encoder that deterministically propagates information upward via a deep neural network, followed by a downward generative model.
The efficacy and scalability of our models are demonstrated on both unsupervised and supervised learning tasks on big corpora.
arXiv Detail & Related papers (2020-06-15T22:22:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.