Towards characterizing the value of edge embeddings in Graph Neural Networks

• URL: http://arxiv.org/abs/2410.09867v1
• Date: Sun, 13 Oct 2024 15:03:24 GMT
• Title: Towards characterizing the value of edge embeddings in Graph Neural Networks
• Authors: Dhruv Rohatgi, Tanya Marwah, Zachary Chase Lipton, Jianfeng Lu, Ankur Moitra, Andrej Risteski,
• Abstract summary: Graph neural networks (GNNs) are the dominant approach to solving machine learning problems defined over graphs. We consider the benefits of architectures that maintain and update edge embeddings.
• Score: 47.01312576529216
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph neural networks (GNNs) are the dominant approach to solving machine learning problems defined over graphs. Despite much theoretical and empirical work in recent years, our understanding of finer-grained aspects of architectural design for GNNs remains impoverished. In this paper, we consider the benefits of architectures that maintain and update edge embeddings. On the theoretical front, under a suitable computational abstraction for a layer in the model, as well as memory constraints on the embeddings, we show that there are natural tasks on graphical models for which architectures leveraging edge embeddings can be much shallower. Our techniques are inspired by results on time-space tradeoffs in theoretical computer science. Empirically, we show architectures that maintain edge embeddings almost always improve on their node-based counterparts -- frequently significantly so in topologies that have ``hub'' nodes.

Related papers

• NN-Former: Rethinking Graph Structure in Neural Architecture Representation [67.3378579108611]
  Graph Neural Networks (GNNs) and transformers have shown promising performance in representing neural architectures.<n>We show that sibling nodes are pivotal while overlooked in previous research.<n>Our approach consistently achieves promising performance in both accuracy and latency prediction.
  arXiv  Detail & Related papers  (2025-07-01T15:46:18Z)
• Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework [10.470880055362406]
  We introduce copresheaf topological neural networks (CTNNs), a powerful and unifying framework that encapsulates a wide spectrum of deep learning architectures.<n>CTNNs address the principled design of neural architectures tailored to specific tasks and data types.<n>Our empirical results on structured data benchmarks demonstrate that CTNNs consistently outperform conventional baselines.
  arXiv  Detail & Related papers  (2025-05-27T14:28:50Z)
• Foundations and Frontiers of Graph Learning Theory [81.39078977407719]
  Recent advancements in graph learning have revolutionized the way to understand and analyze data with complex structures. Graph Neural Networks (GNNs), i.e. neural network architectures designed for learning graph representations, have become a popular paradigm. This article provides a comprehensive summary of the theoretical foundations and breakthroughs concerning the approximation and learning behaviors intrinsic to prevalent graph learning models.
  arXiv  Detail & Related papers  (2024-07-03T14:07:41Z)
• MAgNET: A Graph U-Net Architecture for Mesh-Based Simulations [0.5185522256407782]
  We present MAgNET, which extends the well-known convolutional neural networks to accommodate arbitrary graph-structured data. We demonstrate the predictive capabilities of MAgNET in surrogate modeling for non-linear finite element simulations in the mechanics of solids.
  arXiv  Detail & Related papers  (2022-11-01T19:23:45Z)
• Learning through structure: towards deep neuromorphic knowledge graph embeddings [0.5906031288935515]
  We propose a strategy to map deep graph learning architectures for knowledge graph reasoning to neuromorphic architectures. Based on the insight that randomly and untrained graph neural networks are able to preserve local graph structures, we compose a frozen neural network shallow knowledge graph embedding models. We experimentally show that already on conventional computing hardware, this leads to a significant speedup and memory reduction while maintaining a competitive performance level.
  arXiv  Detail & Related papers  (2021-09-21T18:01:04Z)
• Graph Neural Networks for Graph Drawing [17.983238300054527]
  We propose a novel framework for the development of Graph Neural Drawers (GND) GNDs rely on neural computation for constructing efficient and complex maps. We prove that this mechanism can be guided by loss functions computed by means of Feedforward Neural Networks.
  arXiv  Detail & Related papers  (2021-09-21T09:58:02Z)
• Edge-featured Graph Neural Architecture Search [131.4361207769865]
  We propose Edge-featured Graph Neural Architecture Search to find the optimal GNN architecture. Specifically, we design rich entity and edge updating operations to learn high-order representations. We show EGNAS can search better GNNs with higher performance than current state-of-the-art human-designed and searched-based GNNs.
  arXiv  Detail & Related papers  (2021-09-03T07:53:18Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)
• Geometrically Principled Connections in Graph Neural Networks [66.51286736506658]
  We argue geometry should remain the primary driving force behind innovation in the emerging field of geometric deep learning. We relate graph neural networks to widely successful computer graphics and data approximation models: radial basis functions (RBFs) We introduce affine skip connections, a novel building block formed by combining a fully connected layer with any graph convolution operator.
  arXiv  Detail & Related papers  (2020-04-06T13:25:46Z)
• Node Masking: Making Graph Neural Networks Generalize and Scale Better [71.51292866945471]
  Graph Neural Networks (GNNs) have received a lot of interest in the recent times. In this paper, we utilize some theoretical tools to better visualize the operations performed by state of the art spatial GNNs. We introduce a simple concept, Node Masking, that allows them to generalize and scale better.
  arXiv  Detail & Related papers  (2020-01-17T06:26:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.