What makes a good feedforward computational graph?

• URL: http://arxiv.org/abs/2502.06751v1
• Date: Mon, 10 Feb 2025 18:26:40 GMT
• Title: What makes a good feedforward computational graph?
• Authors: Alex Vitvitskyi, João G. M. Araújo, Marc Lackenby, Petar Veličković,
• Abstract summary: We study desirable properties of a feedforward computational graph, discovering two important complementary measures: fidelity and mixing time. Our study is backed by both theoretical analyses of the metrics' behaviour for various graphs, as well as correlating these metrics to the performance of trained neural network models.
• Score: 0.8370225749625163
• License:
• Abstract: As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects related to the computational graph, such as under-reaching and over-squashing, may even render the model incapable of learning certain functions. Most of these effects have only been thoroughly studied in the domain of undirected graphs; however, recent years have seen a significant rise in interest in feedforward computational graphs: directed graphs without any back edges. In this paper, we study the desirable properties of a feedforward computational graph, discovering two important complementary measures: fidelity and mixing time, and evaluating a few popular choices of graphs through the lens of these measures. Our study is backed by both theoretical analyses of the metrics' asymptotic behaviour for various graphs, as well as correlating these metrics to the performance of trained neural network models using the corresponding graphs.

Related papers

• Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis [7.309233340654514]
  This paper introduces a theoretical framework that rigorously analyzes graph prompting from a data operation perspective. We provide a formal guarantee theorem, demonstrating graph prompts capacity to approximate graph transformation operators. We derive upper bounds on the error of these data operations by graph prompts for a single graph and extend this discussion to batches of graphs.
  arXiv  Detail & Related papers  (2024-10-02T15:07:13Z)
• State of the Art and Potentialities of Graph-level Learning [54.68482109186052]
  Graph-level learning has been applied to many tasks including comparison, regression, classification, and more. Traditional approaches to learning a set of graphs rely on hand-crafted features, such as substructures. Deep learning has helped graph-level learning adapt to the growing scale of graphs by extracting features automatically and encoding graphs into low-dimensional representations.
  arXiv  Detail & Related papers  (2023-01-14T09:15:49Z)
• Robust Causal Graph Representation Learning against Confounding Effects [21.380907101361643]
  We propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects. RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders.
  arXiv  Detail & Related papers  (2022-08-18T01:31:25Z)
• Learning node embeddings via summary graphs: a brief theoretical analysis [55.25628709267215]
  Graph representation learning plays an important role in many graph mining applications, but learning embeddings of large-scale graphs remains a problem. Recent works try to improve scalability via graph summarization -- i.e., they learn embeddings on a smaller summary graph, and then restore the node embeddings of the original graph. We give an in-depth theoretical analysis of three specific embedding learning methods based on introduced kernel matrix.
  arXiv  Detail & Related papers  (2022-07-04T04:09:50Z)
• Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications [52.65389473899139]
  Graphs are a powerful tool for representing and analyzing unstructured, non-Euclidean data ubiquitous in the healthcare domain. Recent works have shown that considering relationships between input data samples have a positive regularizing effect for the downstream task. We propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications.
  arXiv  Detail & Related papers  (2022-04-01T10:01:37Z)
• Graph Self-supervised Learning with Accurate Discrepancy Learning [64.69095775258164]
  We propose a framework that aims to learn the exact discrepancy between the original and the perturbed graphs, coined as Discrepancy-based Self-supervised LeArning (D-SLA) We validate our method on various graph-related downstream tasks, including molecular property prediction, protein function prediction, and link prediction tasks, on which our model largely outperforms relevant baselines.
  arXiv  Detail & Related papers  (2022-02-07T08:04:59Z)
• Graph-wise Common Latent Factor Extraction for Unsupervised Graph Representation Learning [40.70562886682939]
  We propose a new principle for unsupervised graph representation learning: Graph-wise Common latent Factor EXtraction (GCFX) GCFX explicitly extract common latent factors from an input graph and achieve improved results on downstream tasks to the current state-of-the-art. Through extensive experiments and analysis, we demonstrate that GCFX is beneficial for graph-level tasks to alleviate distractions caused by local variations of individual nodes or local neighbourhoods.
  arXiv  Detail & Related papers  (2021-12-16T12:22:49Z)
• Graph Coarsening with Neural Networks [8.407217618651536]
  We propose a framework for measuring the quality of coarsening algorithm and show that depending on the goal, we need to carefully choose the Laplace operator on the coarse graph. Motivated by the observation that the current choice of edge weight for the coarse graph may be sub-optimal, we parametrize the weight assignment map with graph neural networks and train it to improve the coarsening quality in an unsupervised way.
  arXiv  Detail & Related papers  (2021-02-02T06:50:07Z)
• Understanding Coarsening for Embedding Large-Scale Graphs [3.6739949215165164]
  Proper analysis of graphs with Machine Learning (ML) algorithms has the potential to yield far-reaching insights into many areas of research and industry. The irregular structure of graph data constitutes an obstacle for running ML tasks on graphs. We analyze the impact of the coarsening quality on the embedding performance both in terms of speed and accuracy.
  arXiv  Detail & Related papers  (2020-09-10T15:06:33Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)
• Deep Learning for Learning Graph Representations [58.649784596090385]
  Mining graph data has become a popular research topic in computer science. The huge amount of network data has posed great challenges for efficient analysis. This motivates the advent of graph representation which maps the graph into a low-dimension vector space.
  arXiv  Detail & Related papers  (2020-01-02T02:13:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.