Dimensionality Reduction Meets Message Passing for Graph Node Embeddings

• URL: http://arxiv.org/abs/2202.00408v2
• Date: Wed, 2 Feb 2022 08:45:40 GMT
• Title: Dimensionality Reduction Meets Message Passing for Graph Node Embeddings
• Authors: Krzysztof Sadowski, Micha{\l} Szarmach, Eddie Mattia
• Abstract summary: We propose PCAPass, a method which combines Principal Component Analysis (PCA) and message passing for generating node embeddings in an unsupervised manner. We show empirically that this approach provides competitive performance compared to popular GNNs on node classification benchmarks. Our research demonstrates that applying dimensionality reduction with message passing and skip connections is a promising mechanism for aggregating long-range dependencies in graph structured data.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Neural Networks (GNNs) have become a popular approach for various applications, ranging from social network analysis to modeling chemical properties of molecules. While GNNs often show remarkable performance on public datasets, they can struggle to learn long-range dependencies in the data due to over-smoothing and over-squashing tendencies. To alleviate this challenge, we propose PCAPass, a method which combines Principal Component Analysis (PCA) and message passing for generating node embeddings in an unsupervised manner and leverages gradient boosted decision trees for classification tasks. We show empirically that this approach provides competitive performance compared to popular GNNs on node classification benchmarks, while gathering information from longer distance neighborhoods. Our research demonstrates that applying dimensionality reduction with message passing and skip connections is a promising mechanism for aggregating long-range dependencies in graph structured data.

Related papers

• Over-Squashing in Graph Neural Networks: A Comprehensive survey [0.0]
  This survey delves into the challenge of over-squashing in Graph Neural Networks (GNNs) It comprehensively explores the causes, consequences, and mitigation strategies for over-squashing. Various methodologies are reviewed, including graph rewiring, novel normalization, spectral analysis, and curvature-based strategies. The survey also discusses the interplay between over-squashing and other GNN limitations, such as over-smoothing.
  arXiv  Detail & Related papers  (2023-08-29T18:46:15Z)
• DEGREE: Decomposition Based Explanation For Graph Neural Networks [55.38873296761104]
  We propose DEGREE to provide a faithful explanation for GNN predictions. By decomposing the information generation and aggregation mechanism of GNNs, DEGREE allows tracking the contributions of specific components of the input graph to the final prediction. We also design a subgraph level interpretation algorithm to reveal complex interactions between graph nodes that are overlooked by previous methods.
  arXiv  Detail & Related papers  (2023-05-22T10:29:52Z)
• Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph [58.99478502486377]
  We propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections.
  arXiv  Detail & Related papers  (2022-09-23T05:24:18Z)
• Detecting Communities from Heterogeneous Graphs: A Context Path-based Graph Neural Network Model [23.525079144108567]
  We build a Context Path-based Graph Neural Network (CP-GNN) model. It embeds the high-order relationship between nodes into the node embedding. It outperforms the state-of-the-art community detection methods.
  arXiv  Detail & Related papers  (2021-09-05T12:28:00Z)
• Explicit Pairwise Factorized Graph Neural Network for Semi-Supervised Node Classification [59.06717774425588]
  We propose the Explicit Pairwise Factorized Graph Neural Network (EPFGNN), which models the whole graph as a partially observed Markov Random Field. It contains explicit pairwise factors to model output-output relations and uses a GNN backbone to model input-output relations. We conduct experiments on various datasets, which shows that our model can effectively improve the performance for semi-supervised node classification on graphs.
  arXiv  Detail & Related papers  (2021-07-27T19:47:53Z)
• Hierarchical graph neural nets can capture long-range interactions [8.067880298298185]
  We study hierarchical message passing models that leverage a multi-resolution representation of a given graph. This facilitates learning of features that span large receptive fields without loss of local information. We introduce Hierarchical Graph Net (HGNet), which for any two connected nodes guarantees existence of message-passing paths of at most logarithmic length.
  arXiv  Detail & Related papers  (2021-07-15T16:24:22Z)
• Schema-Aware Deep Graph Convolutional Networks for Heterogeneous Graphs [10.526065883783899]
  Graph convolutional network (GCN) based approaches have achieved significant progress for solving complex, graph-structured problems. We propose our GCN framework 'Deep Heterogeneous Graph Convolutional Network (DHGCN)' It takes advantage of the schema of a heterogeneous graph and uses a hierarchical approach to effectively utilize information many hops away.
  arXiv  Detail & Related papers  (2021-05-03T06:24:27Z)
• QD-GCN: Query-Driven Graph Convolutional Networks for Attributed Community Search [54.42038098426504]
  QD-GCN is an end-to-end framework that unifies the community structure as well as node attributes to solve the ACS problem. We show that QD-GCN outperforms existing attributed community search algorithms in terms of both efficiency and effectiveness.
  arXiv  Detail & Related papers  (2021-04-08T07:52:48Z)
• Distance-aware Molecule Graph Attention Network for Drug-Target Binding Affinity Prediction [54.93890176891602]
  We propose a diStance-aware Molecule graph Attention Network (S-MAN) tailored to drug-target binding affinity prediction. As a dedicated solution, we first propose a position encoding mechanism to integrate the topological structure and spatial position information into the constructed pocket-ligand graph. We also propose a novel edge-node hierarchical attentive aggregation structure which has edge-level aggregation and node-level aggregation.
  arXiv  Detail & Related papers  (2020-12-17T17:44:01Z)
• Policy-GNN: Aggregation Optimization for Graph Neural Networks [60.50932472042379]
  Graph neural networks (GNNs) aim to model the local graph structures and capture the hierarchical patterns by aggregating the information from neighbors. It is a challenging task to develop an effective aggregation strategy for each node, given complex graphs and sparse features. We propose Policy-GNN, a meta-policy framework that models the sampling procedure and message passing of GNNs into a combined learning process.
  arXiv  Detail & Related papers  (2020-06-26T17:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.