Permutation invariant matrix statistics and computational language tasks
- URL: http://arxiv.org/abs/2202.06829v2
- Date: Tue, 26 Sep 2023 17:29:38 GMT
- Title: Permutation invariant matrix statistics and computational language tasks
- Authors: Manuel Accettulli Huber, Adriana Correia, Sanjaye Ramgoolam, Mehrnoosh
Sadrzadeh
- Abstract summary: We introduce a geometry of observable vectors for words, defined by exploiting the graph-theoretic basis for the permutation invariants.
We describe successful applications of this unified framework to a number of tasks in computational linguistics.
- Score: 0.7373617024876724
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Linguistic Matrix Theory programme introduced by Kartsaklis, Ramgoolam
and Sadrzadeh is an approach to the statistics of matrices that are generated
in type-driven distributional semantics, based on permutation invariant
polynomial functions which are regarded as the key observables encoding the
significant statistics. In this paper we generalize the previous results on the
approximate Gaussianity of matrix distributions arising from compositional
distributional semantics. We also introduce a geometry of observable vectors
for words, defined by exploiting the graph-theoretic basis for the permutation
invariants and the statistical characteristics of the ensemble of matrices
associated with the words. We describe successful applications of this unified
framework to a number of tasks in computational linguistics, associated with
the distinctions between synonyms, antonyms, hypernyms and hyponyms.
Related papers
- Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.
arXiv Detail & Related papers (2024-07-15T07:11:44Z) - Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
We show that PCA becomes computationally hard at a critical value of the signal's magnitude.
We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree.
It also lets us analyze a new problem of distinguishing between different ensembles.
arXiv Detail & Related papers (2024-04-29T14:33:24Z) - Domain Embeddings for Generating Complex Descriptions of Concepts in
Italian Language [65.268245109828]
We propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries.
The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface.
Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge.
arXiv Detail & Related papers (2024-02-26T15:04:35Z) - Mathematical Foundations for a Compositional Account of the Bayesian
Brain [0.0]
We use the tools of contemporary applied category theory to supply functorial semantics for approximate inference.
We define fibrations of statistical games and classify various problems of statistical inference as corresponding sections.
We construct functors which explain the compositional structure of predictive coding neural circuits under the free energy principle.
arXiv Detail & Related papers (2022-12-23T18:58:17Z) - Connecting Permutation Equivariant Neural Networks and Partition
Diagrams [0.0]
We show how the Schur-Weyl duality that exists between the partition algebra and the symmetric group results in a stronger theoretical foundation for characterising possible permutation equivariant neural networks.
In particular, we find a basis of matrices for the learnable, linear, permutation equivariant layer functions between such tensor power spaces in the standard basis of $M_n$.
arXiv Detail & Related papers (2022-12-16T18:48:54Z) - Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints.
The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution.
We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
arXiv Detail & Related papers (2022-10-21T13:19:45Z) - Adversarially-Trained Nonnegative Matrix Factorization [77.34726150561087]
We consider an adversarially-trained version of the nonnegative matrix factorization.
In our formulation, an attacker adds an arbitrary matrix of bounded norm to the given data matrix.
We design efficient algorithms inspired by adversarial training to optimize for dictionary and coefficient matrices.
arXiv Detail & Related papers (2021-04-10T13:13:17Z) - Joint Network Topology Inference via Structured Fusion Regularization [70.30364652829164]
Joint network topology inference represents a canonical problem of learning multiple graph Laplacian matrices from heterogeneous graph signals.
We propose a general graph estimator based on a novel structured fusion regularization.
We show that the proposed graph estimator enjoys both high computational efficiency and rigorous theoretical guarantee.
arXiv Detail & Related papers (2021-03-05T04:42:32Z) - The general theory of permutation equivarant neural networks and higher
order graph variational encoders [6.117371161379209]
We derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously.
This case arises naturally in graph learning and relation learning applications.
We present a second order graph variational encoder, and show that the latent distribution of equivariant generative models must be exchangeable.
arXiv Detail & Related papers (2020-04-08T13:29:56Z) - Positive maps and trace polynomials from the symmetric group [0.0]
We develop a method to obtain operator inequalities and identities in several variables.
We give connections to concepts in quantum information theory and invariant theory.
arXiv Detail & Related papers (2020-02-28T17:43:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.