Nonparametric Sparse Tensor Factorization with Hierarchical Gamma
Processes
- URL: http://arxiv.org/abs/2110.10082v1
- Date: Tue, 19 Oct 2021 16:17:26 GMT
- Title: Nonparametric Sparse Tensor Factorization with Hierarchical Gamma
Processes
- Authors: Conor Tillinghast, Zheng Wang, Shandian Zhe
- Abstract summary: We propose a nonparametric factorization approach for sparsely observed tensors.
We use hierarchical Gamma processes and Poisson random measures to construct a tensor-valued process.
For efficient inference, we use Dirichlet process properties over finite sample partitions, density transformations, and random features.
- Score: 16.79618682556073
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a nonparametric factorization approach for sparsely observed
tensors. The sparsity does not mean zero-valued entries are massive or
dominated. Rather, it implies the observed entries are very few, and even fewer
with the growth of the tensor; this is ubiquitous in practice. Compared with
the existent works, our model not only leverages the structural information
underlying the observed entry indices, but also provides extra interpretability
and flexibility -- it can simultaneously estimate a set of location factors
about the intrinsic properties of the tensor nodes, and another set of
sociability factors reflecting their extrovert activity in interacting with
others; users are free to choose a trade-off between the two types of factors.
Specifically, we use hierarchical Gamma processes and Poisson random measures
to construct a tensor-valued process, which can freely sample the two types of
factors to generate tensors and always guarantees an asymptotic sparsity. We
then normalize the tensor process to obtain hierarchical Dirichlet processes to
sample each observed entry index, and use a Gaussian process to sample the
entry value as a nonlinear function of the factors, so as to capture both the
sparse structure properties and complex node relationships. For efficient
inference, we use Dirichlet process properties over finite sample partitions,
density transformations, and random features to develop a stochastic
variational estimation algorithm. We demonstrate the advantage of our method in
several benchmark datasets.
Related papers
- Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
We show that PCA becomes computationally hard at a critical value of the signal's magnitude.
We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree.
It also lets us analyze a new problem of distinguishing between different ensembles.
arXiv Detail & Related papers (2024-04-29T14:33:24Z) - Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse
Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
This work introduces a novel principle for disentanglement we call mechanism sparsity regularization.
We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors.
We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
arXiv Detail & Related papers (2024-01-10T02:38:21Z) - Posterior Contraction Rates for Mat\'ern Gaussian Processes on
Riemannian Manifolds [51.68005047958965]
We show that intrinsic Gaussian processes can achieve better performance in practice.
Our work shows that finer-grained analyses are needed to distinguish between different levels of data-efficiency.
arXiv Detail & Related papers (2023-09-19T20:30:58Z) - Estimating Higher-Order Mixed Memberships via the $\ell_{2,\infty}$
Tensor Perturbation Bound [8.521132000449766]
We propose the tensor mixed-membership blockmodel, a generalization of the tensor blockmodel.
We establish the identifiability of our model and propose a computationally efficient estimation procedure.
We apply our methodology to real and simulated data, demonstrating some effects not identifiable from the model with discrete community memberships.
arXiv Detail & Related papers (2022-12-16T18:32:20Z) - Tensor Full Feature Measure and Its Nonconvex Relaxation Applications to
Tensor Recovery [1.8899300124593645]
We propose a new tensor sparsity measure called Full Feature Measure (FFM)
It can simultaneously describe the feature dimension each dimension, and connect the Tucker rank with the tensor tube rank.
Two efficient models based on FFM are proposed, and two Alternating Multiplier Method (ADMM) algorithms are developed to solve the proposed model.
arXiv Detail & Related papers (2021-09-25T01:44:34Z) - Large-Scale Learning with Fourier Features and Tensor Decompositions [3.6930948691311007]
We exploit the tensor product structure of deterministic Fourier features, which enables us to represent the model parameters as a low-rank tensor decomposition.
We demonstrate by means of numerical experiments how our low-rank tensor approach obtains the same performance of the corresponding nonparametric model.
arXiv Detail & Related papers (2021-09-03T14:12:53Z) - Low-rank Characteristic Tensor Density Estimation Part I: Foundations [38.05393186002834]
We propose a novel approach that builds upon tensor factorization tools.
In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor.
We demonstrate the very promising performance of the proposed method using several measured datasets.
arXiv Detail & Related papers (2020-08-27T18:06:19Z) - Provable Online CP/PARAFAC Decomposition of a Structured Tensor via
Dictionary Learning [18.464203215845373]
We consider the problem of factorizing a structured 3-mutation tensor into its constituent Polyadic (CP) factors.
We develop a provable algorithm for structured tensor factorization.
arXiv Detail & Related papers (2020-06-30T00:31:06Z) - Uncertainty quantification for nonconvex tensor completion: Confidence
intervals, heteroscedasticity and optimality [92.35257908210316]
We study the problem of estimating a low-rank tensor given incomplete and corrupted observations.
We find that it attains unimprovable rates $ell-2$ accuracy.
arXiv Detail & Related papers (2020-06-15T17:47:13Z) - Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
empirical optimization is central to modern machine learning, but its role in its success is still unclear.
We show that it commonly arises in parameters of discrete multiplicative noise due to variance.
A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
arXiv Detail & Related papers (2020-06-11T09:58:01Z) - Supervised Learning for Non-Sequential Data: A Canonical Polyadic
Decomposition Approach [85.12934750565971]
Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks.
To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor.
For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
arXiv Detail & Related papers (2020-01-27T22:38:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.