Generalized Simplicial Attention Neural Networks
- URL: http://arxiv.org/abs/2309.02138v1
- Date: Tue, 5 Sep 2023 11:29:25 GMT
- Title: Generalized Simplicial Attention Neural Networks
- Authors: Claudio Battiloro, Lucia Testa, Lorenzo Giusti, Stefania Sardellitti,
Paolo Di Lorenzo, Sergio Barbarossa
- Abstract summary: We introduce Generalized Simplicial Attention Neural Networks (GSANs)
GSANs process data defined on simplicial complexes using masked self-attentional layers.
We theoretically establish that GSANs are permutation equivariant and simplicial-aware.
- Score: 23.493128450672348
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The aim of this work is to introduce Generalized Simplicial Attention Neural
Networks (GSANs), i.e., novel neural architectures designed to process data
defined on simplicial complexes using masked self-attentional layers. Hinging
on topological signal processing principles, we devise a series of
self-attention schemes capable of processing data components defined at
different simplicial orders, such as nodes, edges, triangles, and beyond. These
schemes learn how to weight the neighborhoods of the given topological domain
in a task-oriented fashion, leveraging the interplay among simplices of
different orders through the Dirac operator and its Dirac decomposition. We
also theoretically establish that GSANs are permutation equivariant and
simplicial-aware. Finally, we illustrate how our approach compares favorably
with other methods when applied to several (inductive and transductive) tasks
such as trajectory prediction, missing data imputation, graph classification,
and simplex prediction.
Related papers
- Pooling Strategies for Simplicial Convolutional Networks [18.80397868603073]
The goal of this paper is to introduce pooling strategies for simplicial convolutional neural networks.
Inspired by graph pooling methods, we introduce a general formulation for a simplicial pooling layer.
The general layer is then customized to design four different pooling strategies.
arXiv Detail & Related papers (2022-10-11T14:45:51Z) - Tuning the Geometry of Graph Neural Networks [0.7614628596146599]
spatial graph convolution operators have been heralded as key to the success of Graph Neural Networks (GNNs)
We show that this aggregation operator is in fact tunable, and explicit regimes in which certain choices of operators -- and therefore, embedding geometries -- might be more appropriate.
arXiv Detail & Related papers (2022-07-12T23:28:03Z) - Inducing Gaussian Process Networks [80.40892394020797]
We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points.
The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains.
We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
arXiv Detail & Related papers (2022-04-21T05:27:09Z) - Simplicial Attention Networks [0.0]
We introduce a proper self-attention mechanism able to process data components at different layers.
We learn how to weight both upper and lower neighborhoods of the given topological domain in a totally task-oriented fashion.
The proposed approach compares favorably with other methods when applied to different (inductive and transductive) tasks.
arXiv Detail & Related papers (2022-03-14T20:47:31Z) - Learning Connectivity with Graph Convolutional Networks for
Skeleton-based Action Recognition [14.924672048447338]
We introduce a novel framework for graph convolutional networks that learns the topological properties of graphs.
The design principle of our method is based on the optimization of a constrained objective function.
Experiments conducted on the challenging task of skeleton-based action recognition shows the superiority of the proposed method.
arXiv Detail & Related papers (2021-12-06T19:43:26Z) - Dist2Cycle: A Simplicial Neural Network for Homology Localization [66.15805004725809]
Simplicial complexes can be viewed as high dimensional generalizations of graphs that explicitly encode multi-way ordered relations.
We propose a graph convolutional model for learning functions parametrized by the $k$-homological features of simplicial complexes.
arXiv Detail & Related papers (2021-10-28T14:59:41Z) - Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole.
We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
arXiv Detail & Related papers (2021-01-16T16:18:43Z) - Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images.
In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner.
We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
arXiv Detail & Related papers (2020-07-13T18:05:36Z) - GCN for HIN via Implicit Utilization of Attention and Meta-paths [104.24467864133942]
Heterogeneous information network (HIN) embedding aims to map the structure and semantic information in a HIN to distributed representations.
We propose a novel neural network method via implicitly utilizing attention and meta-paths.
We first use the multi-layer graph convolutional network (GCN) framework, which performs a discriminative aggregation at each layer.
We then give an effective relaxation and improvement via introducing a new propagation operation which can be separated from aggregation.
arXiv Detail & Related papers (2020-07-06T11:09:40Z) - Progressive Graph Convolutional Networks for Semi-Supervised Node
Classification [97.14064057840089]
Graph convolutional networks have been successful in addressing graph-based tasks such as semi-supervised node classification.
We propose a method to automatically build compact and task-specific graph convolutional networks.
arXiv Detail & Related papers (2020-03-27T08:32:16Z) - Investigating the Compositional Structure Of Deep Neural Networks [1.8899300124593645]
We introduce a novel theoretical framework based on the compositional structure of piecewise linear activation functions.
It is possible to characterize the instances of the input data with respect to both the predicted label and the specific (linear) transformation used to perform predictions.
Preliminary tests on the MNIST dataset show that our method can group input instances with regard to their similarity in the internal representation of the neural network.
arXiv Detail & Related papers (2020-02-17T14:16:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.