Consistency of random-walk based network embedding algorithms

• URL: http://arxiv.org/abs/2101.07354v1
• Date: Mon, 18 Jan 2021 22:49:22 GMT
• Title: Consistency of random-walk based network embedding algorithms
• Authors: Yichi Zhang, Minh Tang
• Abstract summary: We study the node2vec and DeepWalk algorithms through the perspective of matrix factorization. Our results indicate a subtle interplay between the sparsity of the observed networks, the window sizes of the random walks, and the convergence rates of the node2vec/DeepWalk embedding.
• Score: 13.214230533788932
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Random-walk based network embedding algorithms like node2vec and DeepWalk are widely used to obtain Euclidean representation of the nodes in a network prior to performing down-stream network inference tasks. Nevertheless, despite their impressive empirical performance, there is a lack of theoretical results explaining their behavior. In this paper we studied the node2vec and DeepWalk algorithms through the perspective of matrix factorization. We analyze these algorithms in the setting of community detection for stochastic blockmodel graphs; in particular we established large-sample error bounds and prove consistent community recovery of node2vec/DeepWalk embedding followed by k-means clustering. Our theoretical results indicate a subtle interplay between the sparsity of the observed networks, the window sizes of the random walks, and the convergence rates of the node2vec/DeepWalk embedding toward the embedding of the true but unknown edge probabilities matrix. More specifically, as the network becomes sparser, our results suggest using larger window sizes, or equivalently, taking longer random walks, in order to attain better convergence rate for the resulting embeddings. The paper includes numerical experiments corroborating these observations.

Related papers

• Inference of Causal Networks using a Topological Threshold [0.10241134756773226]
  We propose a constraint-based algorithm, which automatically determines causal relevance thresholds. We show that this novel algorithm is generally faster and more accurate than the PC algorithm.
  arXiv  Detail & Related papers  (2024-04-21T21:56:39Z)
• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• NODDLE: Node2vec based deep learning model for link prediction [0.0]
  We propose NODDLE (integration of NOde2vec anD Deep Learning mEthod), a deep learning model which incorporates the features extracted by node2vec and feeds them into a hidden neural network. Experimental results show that this method yields better results than the traditional methods on various social network datasets.
  arXiv  Detail & Related papers  (2023-05-25T18:43:52Z)
• Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis [93.55896765176414]
  We propose binary sparse convolutional networks called BSC-Net for efficient point cloud analysis. We employ the differentiable search strategies to discover the optimal opsitions for active site matching in the shifted sparse convolution. Our BSC-Net achieves significant improvement upon our srtong baseline and outperforms the state-of-the-art network binarization methods.
  arXiv  Detail & Related papers  (2023-03-27T13:47:06Z)
• Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach [64.23331120621118]
  This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks. We introduce a related embedded network and show that the embedded network can be used to provide an $ell_infty$-norm box over-approximation of the reachable sets of the original network. We apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.
  arXiv  Detail & Related papers  (2022-08-08T03:13:24Z)
• Interpolation-based Correlation Reduction Network for Semi-Supervised Graph Learning [49.94816548023729]
  We propose a novel graph contrastive learning method, termed Interpolation-based Correlation Reduction Network (ICRN) In our method, we improve the discriminative capability of the latent feature by enlarging the margin of decision boundaries. By combining the two settings, we extract rich supervision information from both the abundant unlabeled nodes and the rare yet valuable labeled nodes for discnative representation learning.
  arXiv  Detail & Related papers  (2022-06-06T14:26:34Z)
• Community detection using low-dimensional network embedding algorithms [1.052782170493037]
  We rigorously understand the performance of two major algorithms, DeepWalk and node2vec, in recovering communities for canonical network models. We prove that, given some fixed co-occurrence window, node2vec using random walks with a low non-backtracking probability can succeed for much sparser networks.
  arXiv  Detail & Related papers  (2021-11-04T14:57:43Z)
• Ergodic Limits, Relaxations, and Geometric Properties of Random Walk Node Embeddings [11.549910517683085]
  Random walk based node embedding algorithms learn vector representations of nodes by optimizing an objective function of node embedding vectors and skip-bigram statistics computed from random walks on the network. This paper studies properties of random walk based node embeddings in the unsupervised setting of discovering hidden block structure in the network.
  arXiv  Detail & Related papers  (2021-09-09T19:24:35Z)
• Overlapping community detection in networks via sparse spectral decomposition [1.0660480034605242]
  We consider the problem of estimating overlapping community memberships in a network, where each node can belong to multiple communities. Our algorithm is based on sparse principal subspace estimation with iterative thresholding. We show that a fixed point of the algorithm corresponds to correct node memberships under a version of the block model.
  arXiv  Detail & Related papers  (2020-09-20T07:31:09Z)
• ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
  We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks. Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
  arXiv  Detail & Related papers  (2020-06-28T23:09:27Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.