Local2Global: Scaling global representation learning on graphs via local training

• URL: http://arxiv.org/abs/2107.12224v1
• Date: Mon, 26 Jul 2021 14:08:31 GMT
• Title: Local2Global: Scaling global representation learning on graphs via local training
• Authors: Lucas G. S. Jeub, Giovanni Colavizza, Xiaowen Dong, Marya Bazzi, Mihai Cucuringu
• Abstract summary: We propose a decentralised "local2global" approach to graph representation learning. We train local representations for each patch independently and combine the local representations into a globally consistent representation. Preliminary results on medium-scale data sets are promising, with a graph reconstruction performance for local2global that is comparable to that of globally trained embeddings.
• Score: 6.292766967410996
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a decentralised "local2global" approach to graph representation learning, that one can a-priori use to scale any embedding technique. Our local2global approach proceeds by first dividing the input graph into overlapping subgraphs (or "patches") and training local representations for each patch independently. In a second step, we combine the local representations into a globally consistent representation by estimating the set of rigid motions that best align the local representations using information from the patch overlaps, via group synchronization. A key distinguishing feature of local2global relative to existing work is that patches are trained independently without the need for the often costly parameter synchronisation during distributed training. This allows local2global to scale to large-scale industrial applications, where the input graph may not even fit into memory and may be stored in a distributed manner. Preliminary results on medium-scale data sets (up to $\sim$7K nodes and $\sim$200K edges) are promising, with a graph reconstruction performance for local2global that is comparable to that of globally trained embeddings. A thorough evaluation of local2global on large scale data and applications to downstream tasks, such as node classification and link prediction, constitutes ongoing work.

Related papers

• L2G2G: a Scalable Local-to-Global Network Embedding with Graph Autoencoders [6.945992777272943]
  graph representation learning is a popular tool for analysing real-world networks. GAEs tend to be fairly accurate, but they suffer from scalability issues. For improved speed, a Local2Global approach was shown to be fast and achieve good accuracy. Here we propose L2G2G, a Local2Global method which improves GAE accuracy without sacrificing scalability.
  arXiv  Detail & Related papers  (2024-02-02T18:24:37Z)
• Balancing between the Local and Global Structures (LGS) in Graph Embedding [1.4732811715354455]
  We present a method for balancing between the Local and Global Structures (LGS) in graph embedding, via a tunable parameter. We evaluate the performance of LGS with synthetic and real-world datasets and our results indicate that it is competitive with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-31T02:12:46Z)
• Federated Learning with Intermediate Representation Regularization [14.01585596739954]
  Federated learning (FL) enables remote clients to collaboratively train a model without exposing their private data. Previous studies accomplish this by regularizing the distance between the representations learned by the local and global models. We introduce FedIntR, which provides a more fine-grained regularization by integrating the representations of intermediate layers into the local training process.
  arXiv  Detail & Related papers  (2022-10-28T01:43:55Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Local2Global: A distributed approach for scaling representation learning on graphs [10.254620252788776]
  We propose a decentralised "local2global"' approach to graph representation learning, that one can a-priori use to scale any embedding technique. We show that our approach achieves a good trade-off between scale and accuracy on edge reconstruction and semi-supervised classification. We also consider the downstream task of anomaly detection and show how one can use local2global to highlight anomalies in cybersecurity networks.
  arXiv  Detail & Related papers  (2022-01-12T23:00:22Z)
• Tackling the Local Bias in Federated Graph Learning [48.887310972708036]
  In Federated graph learning (FGL), a global graph is distributed across different clients, where each client holds a subgraph. Existing FGL methods fail to effectively utilize cross-client edges, losing structural information during the training. We propose a novel FGL framework to make the local models similar to the model trained in a centralized setting.
  arXiv  Detail & Related papers  (2021-10-22T08:22:36Z)
• Self-supervised Graph-level Representation Learning with Local and Global Structure [71.45196938842608]
  We propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model.
  arXiv  Detail & Related papers  (2021-06-08T05:25:38Z)
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation [87.50205728818601]
  We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space. Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
  arXiv  Detail & Related papers  (2020-11-25T11:03:11Z)
• Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global Localization in Large Scale Environment [18.128244946109795]
  We propose a semantic histogram-based graph matching method that is robust to viewpoint variation and can achieve real-time global localization. Our approach is about 30 times faster than Random Walk based semantic descriptors. It achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.
  arXiv  Detail & Related papers  (2020-10-19T08:18:42Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)
• Think Locally, Act Globally: Federated Learning with Local and Global Representations [92.68484710504666]
  Federated learning is a method of training models on private data distributed over multiple devices. We propose a new federated learning algorithm that jointly learns compact local representations on each device. We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key.
  arXiv  Detail & Related papers  (2020-01-06T12:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.