Local2Global: A distributed approach for scaling representation learning on graphs

• URL: http://arxiv.org/abs/2201.04729v1
• Date: Wed, 12 Jan 2022 23:00:22 GMT
• Title: Local2Global: A distributed approach for scaling representation learning on graphs
• Authors: Lucas G. S. Jeub, Giovanni Colavizza, Xiaowen Dong, Marya Bazzi, Mihai Cucuringu
• Abstract summary: We propose a decentralised "local2global"' approach to graph representation learning, that one can a-priori use to scale any embedding technique. We show that our approach achieves a good trade-off between scale and accuracy on edge reconstruction and semi-supervised classification. We also consider the downstream task of anomaly detection and show how one can use local2global to highlight anomalies in cybersecurity networks.
• Score: 10.254620252788776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a decentralised "local2global"' approach to graph representation learning, that one can a-priori use to scale any embedding technique. Our local2global approach proceeds by first dividing the input graph into overlapping subgraphs (or "patches") and training local representations for each patch independently. In a second step, we combine the local representations into a globally consistent representation by estimating the set of rigid motions that best align the local representations using information from the patch overlaps, via group synchronization. A key distinguishing feature of local2global relative to existing work is that patches are trained independently without the need for the often costly parameter synchronization during distributed training. This allows local2global to scale to large-scale industrial applications, where the input graph may not even fit into memory and may be stored in a distributed manner. We apply local2global on data sets of different sizes and show that our approach achieves a good trade-off between scale and accuracy on edge reconstruction and semi-supervised classification. We also consider the downstream task of anomaly detection and show how one can use local2global to highlight anomalies in cybersecurity networks.

Related papers

• FUSELOC: Fusing Global and Local Descriptors to Disambiguate 2D-3D Matching in Visual Localization [57.59857784298536]
  Direct 2D-3D matching algorithms require significantly less memory but suffer from lower accuracy due to the larger and more ambiguous search space. We address this ambiguity by fusing local and global descriptors using a weighted average operator within a 2D-3D search framework. We consistently improve the accuracy over local-only systems and achieve performance close to hierarchical methods while halving memory requirements.
  arXiv  Detail & Related papers  (2024-08-21T23:42:16Z)
• Balancing between the Local and Global Structures (LGS) in Graph Embedding [1.4732811715354455]
  We present a method for balancing between the Local and Global Structures (LGS) in graph embedding, via a tunable parameter. We evaluate the performance of LGS with synthetic and real-world datasets and our results indicate that it is competitive with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-31T02:12:46Z)
• Federated Learning with Intermediate Representation Regularization [14.01585596739954]
  Federated learning (FL) enables remote clients to collaboratively train a model without exposing their private data. Previous studies accomplish this by regularizing the distance between the representations learned by the local and global models. We introduce FedIntR, which provides a more fine-grained regularization by integrating the representations of intermediate layers into the local training process.
  arXiv  Detail & Related papers  (2022-10-28T01:43:55Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Global-Local Self-Distillation for Visual Representation Learning [41.24728444810133]
  Richer and more meaningful gradients updates are key to allow self-supervised methods to learn better and in a more efficient manner. In a typical self-distillation framework, the representation of two augmented images are enforced to be coherent at the global level. We propose to leverage the spatial information in the input images to obtain geometric matchings.
  arXiv  Detail & Related papers  (2022-07-29T13:50:09Z)
• Tackling the Local Bias in Federated Graph Learning [48.887310972708036]
  In Federated graph learning (FGL), a global graph is distributed across different clients, where each client holds a subgraph. Existing FGL methods fail to effectively utilize cross-client edges, losing structural information during the training. We propose a novel FGL framework to make the local models similar to the model trained in a centralized setting.
  arXiv  Detail & Related papers  (2021-10-22T08:22:36Z)
• Local2Global: Scaling global representation learning on graphs via local training [6.292766967410996]
  We propose a decentralised "local2global" approach to graph representation learning. We train local representations for each patch independently and combine the local representations into a globally consistent representation. Preliminary results on medium-scale data sets are promising, with a graph reconstruction performance for local2global that is comparable to that of globally trained embeddings.
  arXiv  Detail & Related papers  (2021-07-26T14:08:31Z)
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation [87.50205728818601]
  We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space. Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
  arXiv  Detail & Related papers  (2020-11-25T11:03:11Z)
• Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global Localization in Large Scale Environment [18.128244946109795]
  We propose a semantic histogram-based graph matching method that is robust to viewpoint variation and can achieve real-time global localization. Our approach is about 30 times faster than Random Walk based semantic descriptors. It achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.
  arXiv  Detail & Related papers  (2020-10-19T08:18:42Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)
• Think Locally, Act Globally: Federated Learning with Local and Global Representations [92.68484710504666]
  Federated learning is a method of training models on private data distributed over multiple devices. We propose a new federated learning algorithm that jointly learns compact local representations on each device. We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key.
  arXiv  Detail & Related papers  (2020-01-06T12:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.