Local2Global: A distributed approach for scaling representation learning
on graphs
- URL: http://arxiv.org/abs/2201.04729v1
- Date: Wed, 12 Jan 2022 23:00:22 GMT
- Title: Local2Global: A distributed approach for scaling representation learning
on graphs
- Authors: Lucas G. S. Jeub, Giovanni Colavizza, Xiaowen Dong, Marya Bazzi, Mihai
Cucuringu
- Abstract summary: We propose a decentralised "local2global"' approach to graph representation learning, that one can a-priori use to scale any embedding technique.
We show that our approach achieves a good trade-off between scale and accuracy on edge reconstruction and semi-supervised classification.
We also consider the downstream task of anomaly detection and show how one can use local2global to highlight anomalies in cybersecurity networks.
- Score: 10.254620252788776
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a decentralised "local2global"' approach to graph representation
learning, that one can a-priori use to scale any embedding technique. Our
local2global approach proceeds by first dividing the input graph into
overlapping subgraphs (or "patches") and training local representations for
each patch independently. In a second step, we combine the local
representations into a globally consistent representation by estimating the set
of rigid motions that best align the local representations using information
from the patch overlaps, via group synchronization. A key distinguishing
feature of local2global relative to existing work is that patches are trained
independently without the need for the often costly parameter synchronization
during distributed training. This allows local2global to scale to large-scale
industrial applications, where the input graph may not even fit into memory and
may be stored in a distributed manner. We apply local2global on data sets of
different sizes and show that our approach achieves a good trade-off between
scale and accuracy on edge reconstruction and semi-supervised classification.
We also consider the downstream task of anomaly detection and show how one can
use local2global to highlight anomalies in cybersecurity networks.
Related papers
- FedCRL: Personalized Federated Learning with Contrastive Shared Representations for Label Heterogeneity in Non-IID Data [13.146806294562474]
This paper proposes a novel personalized Federated Learning algorithm, named Federated Contrastive Learning (FedCRL)
FedCRL introduces contrastive representation learning on shared representations to facilitate knowledge acquisition of clients.
Our simulations demonstrate FedCRL's effectiveness in mitigating label heterogeneity by achieving accuracy improvements over existing methods.
arXiv Detail & Related papers (2024-04-27T14:05:18Z) - Balancing between the Local and Global Structures (LGS) in Graph
Embedding [1.4732811715354455]
We present a method for balancing between the Local and Global Structures (LGS) in graph embedding, via a tunable parameter.
We evaluate the performance of LGS with synthetic and real-world datasets and our results indicate that it is competitive with the state-of-the-art methods.
arXiv Detail & Related papers (2023-08-31T02:12:46Z) - Federated Learning with Intermediate Representation Regularization [14.01585596739954]
Federated learning (FL) enables remote clients to collaboratively train a model without exposing their private data.
Previous studies accomplish this by regularizing the distance between the representations learned by the local and global models.
We introduce FedIntR, which provides a more fine-grained regularization by integrating the representations of intermediate layers into the local training process.
arXiv Detail & Related papers (2022-10-28T01:43:55Z) - Adaptive Local-Component-aware Graph Convolutional Network for One-shot
Skeleton-based Action Recognition [54.23513799338309]
We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition.
Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
arXiv Detail & Related papers (2022-09-21T02:33:07Z) - Federated and Generalized Person Re-identification through Domain and
Feature Hallucinating [88.77196261300699]
We study the problem of federated domain generalization (FedDG) for person re-identification (re-ID)
We propose a novel method, called "Domain and Feature Hallucinating (DFH)", to produce diverse features for learning generalized local and global models.
Our method achieves the state-of-the-art performance for FedDG on four large-scale re-ID benchmarks.
arXiv Detail & Related papers (2022-03-05T09:15:13Z) - Local2Global: Scaling global representation learning on graphs via local
training [6.292766967410996]
We propose a decentralised "local2global" approach to graph representation learning.
We train local representations for each patch independently and combine the local representations into a globally consistent representation.
Preliminary results on medium-scale data sets are promising, with a graph reconstruction performance for local2global that is comparable to that of globally trained embeddings.
arXiv Detail & Related papers (2021-07-26T14:08:31Z) - PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image
Segmentation [87.50205728818601]
We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space.
Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
arXiv Detail & Related papers (2020-11-25T11:03:11Z) - Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
Training networks to perform metric relocalization traditionally requires accurate image correspondences.
We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration.
We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
arXiv Detail & Related papers (2020-11-01T19:24:27Z) - Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global
Localization in Large Scale Environment [18.128244946109795]
We propose a semantic histogram-based graph matching method that is robust to viewpoint variation and can achieve real-time global localization.
Our approach is about 30 times faster than Random Walk based semantic descriptors.
It achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.
arXiv Detail & Related papers (2020-10-19T08:18:42Z) - Multi-Level Graph Convolutional Network with Automatic Graph Learning
for Hyperspectral Image Classification [63.56018768401328]
We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification.
By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions.
Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
arXiv Detail & Related papers (2020-09-19T09:26:20Z) - Think Locally, Act Globally: Federated Learning with Local and Global
Representations [92.68484710504666]
Federated learning is a method of training models on private data distributed over multiple devices.
We propose a new federated learning algorithm that jointly learns compact local representations on each device.
We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key.
arXiv Detail & Related papers (2020-01-06T12:40:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.