Neural Nonnegative Matrix Factorization for Hierarchical Multilayer
Topic Modeling
- URL: http://arxiv.org/abs/2303.00058v1
- Date: Tue, 28 Feb 2023 20:00:16 GMT
- Title: Neural Nonnegative Matrix Factorization for Hierarchical Multilayer
Topic Modeling
- Authors: Tyler Will, Runyu Zhang, Eli Sadovnik, Mengdi Gao, Joshua Vendrow,
Jamie Haddock, Denali Molitor, Deanna Needell
- Abstract summary: We introduce a new method based on nonnegative matrix factorization, Neural NMF, for detecting latent hierarchical structure in data.
We test Neural NMF on a synthetic hierarchical dataset, the 20 Newsgroups dataset, and the MyLymeData symptoms dataset.
Numerical results demonstrate that Neural NMF outperforms other hierarchical NMF methods on these data sets.
- Score: 7.976416480654409
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce a new method based on nonnegative matrix factorization, Neural
NMF, for detecting latent hierarchical structure in data. Datasets with
hierarchical structure arise in a wide variety of fields, such as document
classification, image processing, and bioinformatics. Neural NMF recursively
applies NMF in layers to discover overarching topics encompassing the
lower-level features. We derive a backpropagation optimization scheme that
allows us to frame hierarchical NMF as a neural network. We test Neural NMF on
a synthetic hierarchical dataset, the 20 Newsgroups dataset, and the MyLymeData
symptoms dataset. Numerical results demonstrate that Neural NMF outperforms
other hierarchical NMF methods on these data sets and offers better learned
hierarchical structure and interpretability of topics.
Related papers
- Defining Neural Network Architecture through Polytope Structures of Dataset [53.512432492636236]
This paper defines upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question.
We develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks.
It is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces.
arXiv Detail & Related papers (2024-02-04T08:57:42Z) - Stratified-NMF for Heterogeneous Data [8.174199227297514]
We propose a modified NMF objective, Stratified-NMF, that simultaneously learns strata-dependent statistics and a shared topics matrix.
We apply our method to three real world datasets and empirically investigate their learned features.
arXiv Detail & Related papers (2023-11-17T00:34:41Z) - Analyzing Single Cell RNA Sequencing with Topological Nonnegative Matrix
Factorization [0.43512163406551996]
Nonnegative matrix factorization (NMF) offers a unique approach due to its meta-gene interpretation of resulting low-dimensional components.
This work introduces two persistent Laplacian regularized NMF methods, namely, topological NMF (TNMF) and robust topological NMF (rTNMF)
By employing a total of 12 datasets, we demonstrate that the proposed TNMF and rTNMF significantly outperform all other NMF-based methods.
arXiv Detail & Related papers (2023-10-24T11:36:41Z) - Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure.
We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance.
We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
arXiv Detail & Related papers (2023-05-24T11:05:12Z) - Multiclass classification for multidimensional functional data through
deep neural networks [0.22843885788439797]
We introduce a novel functional deep neural network (mfDNN) as an innovative data mining classification tool.
We consider sparse deep neural network architecture with linear unit (ReLU) activation function and minimize the cross-entropy loss in the multiclass classification setup.
We demonstrate the performance of mfDNN on simulated data and several benchmark datasets from different application domains.
arXiv Detail & Related papers (2023-05-22T16:56:01Z) - WLD-Reg: A Data-dependent Within-layer Diversity Regularizer [98.78384185493624]
Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization.
We propose to complement this traditional 'between-layer' feedback with additional 'within-layer' feedback to encourage the diversity of the activations within the same layer.
We present an extensive empirical study confirming that the proposed approach enhances the performance of several state-of-the-art neural network models in multiple tasks.
arXiv Detail & Related papers (2023-01-03T20:57:22Z) - SymNMF-Net for The Symmetric NMF Problem [62.44067422984995]
We propose a neural network called SymNMF-Net for the Symmetric NMF problem.
We show that the inference of each block corresponds to a single iteration of the optimization.
Empirical results on real-world datasets demonstrate the superiority of our SymNMF-Net.
arXiv Detail & Related papers (2022-05-26T08:17:39Z) - Multi-Scale Semantics-Guided Neural Networks for Efficient
Skeleton-Based Human Action Recognition [140.18376685167857]
A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition.
MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
arXiv Detail & Related papers (2021-11-07T03:50:50Z) - Dual-constrained Deep Semi-Supervised Coupled Factorization Network with
Enriched Prior [80.5637175255349]
We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net.
To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network.
Our network can obtain state-of-the-art performance for representation learning and clustering.
arXiv Detail & Related papers (2020-09-08T13:10:21Z) - Neighborhood Structure Assisted Non-negative Matrix Factorization and
its Application in Unsupervised Point-wise Anomaly Detection [6.859284479314336]
We propose to incorporate the neighborhood structure information within the NMF framework by modeling the data through a minimum spanning tree.
We label the resulting method as the neighborhood structure assisted NMF.
Empirical comparisons using twenty benchmark datasets as well as an industrial dataset extracted from a hydropower plant demonstrate the superiority of the neighborhood structure assisted NMF.
arXiv Detail & Related papers (2020-01-17T21:43:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.