A Self-Attention Network based Node Embedding Model

• URL: http://arxiv.org/abs/2006.12100v1
• Date: Mon, 22 Jun 2020 09:46:10 GMT
• Title: A Self-Attention Network based Node Embedding Model
• Authors: Dai Quoc Nguyen and Tu Dinh Nguyen and Dinh Phung
• Abstract summary: We propose SANNE -- a novel unsupervised embedding model. Our SANNE aims to produce plausible embeddings not only for present nodes, but also for newly unseen nodes. Experimental results show that the proposed SANNE obtains state-of-the-art results for the node classification task.
• Score: 17.10479440152652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite several signs of progress have been made recently, limited research has been conducted for an inductive setting where embeddings are required for newly unseen nodes -- a setting encountered commonly in practical applications of deep learning for graph networks. This significantly affects the performances of downstream tasks such as node classification, link prediction or community extraction. To this end, we propose SANNE -- a novel unsupervised embedding model -- whose central idea is to employ a transformer self-attention network to iteratively aggregate vector representations of nodes in random walks. Our SANNE aims to produce plausible embeddings not only for present nodes, but also for newly unseen nodes. Experimental results show that the proposed SANNE obtains state-of-the-art results for the node classification task on well-known benchmark datasets.

Related papers

• Global Context Enhanced Anomaly Detection of Cyber Attacks via Decoupled Graph Neural Networks [0.0]
  We deploy decoupled GNNs to overcome the issue of capturing nonlinear network information. For node representation learning, we develop a GNN architecture with two modules for aggregating node feature information. The findings demonstrate that decoupled training along with the global context enhanced representation of the nodes is superior to the state-of-the-art models in terms of AUC.
  arXiv  Detail & Related papers  (2024-09-04T21:54:07Z)
• Reinforcement Learning for Node Selection in Branch-and-Bound [52.2648997215667]
  Current state-of-the-art selectors utilize either hand-crafted ensembles that automatically switch between naive sub-node selectors, or learned node selectors that rely on individual node data. We propose a novel simulation technique that uses reinforcement learning (RL) while considering the entire tree state, rather than just isolated nodes.
  arXiv  Detail & Related papers  (2023-09-29T19:55:56Z)
• Collaborative Graph Neural Networks for Attributed Network Embedding [63.39495932900291]
  Graph neural networks (GNNs) have shown prominent performance on attributed network embedding. We propose COllaborative graph Neural Networks--CONN, a tailored GNN architecture for network embedding.
  arXiv  Detail & Related papers  (2023-07-22T04:52:27Z)
• Hierarchical Prototype Networks for Continual Graph Representation Learning [90.78466005753505]
  We present Hierarchical Prototype Networks (HPNs) which extract different levels of abstract knowledge in the form of prototypes to represent the continuously expanded graphs. We show that HPNs not only outperform state-of-the-art baseline techniques but also consume relatively less memory.
  arXiv  Detail & Related papers  (2021-11-30T14:15:14Z)
• Topic-aware latent models for representation learning on networks [5.304857921982132]
  We introduce TNE, a generic framework to enhance the embeddings of nodes acquired by means of random walk-based approaches with topic-based information. We evaluate our methodology in two downstream tasks: node classification and link prediction.
  arXiv  Detail & Related papers  (2021-11-10T08:52:52Z)
• SSSNET: Semi-Supervised Signed Network Clustering [4.895808607591299]
  We introduce a novel probabilistic balanced normalized cut loss for training nodes in a GNN framework for semi-supervised signed network clustering, called SSSNET. The main novelty approach is a new take on the role of social balance theory for signed network embeddings.
  arXiv  Detail & Related papers  (2021-10-13T10:36:37Z)
• Ergodic Limits, Relaxations, and Geometric Properties of Random Walk Node Embeddings [11.549910517683085]
  Random walk based node embedding algorithms learn vector representations of nodes by optimizing an objective function of node embedding vectors and skip-bigram statistics computed from random walks on the network. This paper studies properties of random walk based node embeddings in the unsupervised setting of discovering hidden block structure in the network.
  arXiv  Detail & Related papers  (2021-09-09T19:24:35Z)
• Node2Seq: Towards Trainable Convolutions in Graph Neural Networks [59.378148590027735]
  We propose a graph network layer, known as Node2Seq, to learn node embeddings with explicitly trainable weights for different neighboring nodes. For a target node, our method sorts its neighboring nodes via attention mechanism and then employs 1D convolutional neural networks (CNNs) to enable explicit weights for information aggregation. In addition, we propose to incorporate non-local information for feature learning in an adaptive manner based on the attention scores.
  arXiv  Detail & Related papers  (2021-01-06T03:05:37Z)
• Integrating Network Embedding and Community Outlier Detection via Multiclass Graph Description [15.679313861083239]
  We propose a novel unsupervised graph embedding approach (called DMGD) which integrates outlier and community detection with node embedding. We show the theoretical bounds on the number of outliers detected by DMGD. Our formulation boils down to an interesting minimax game between the outliers, community assignments and the node embedding function.
  arXiv  Detail & Related papers  (2020-07-20T16:21:07Z)
• Graph Prototypical Networks for Few-shot Learning on Attributed Networks [72.31180045017835]
  We propose a graph meta-learning framework -- Graph Prototypical Networks (GPN) GPN is able to perform textitmeta-learning on an attributed network and derive a highly generalizable model for handling the target classification task.
  arXiv  Detail & Related papers  (2020-06-23T04:13:23Z)
• Benchmarking Network Embedding Models for Link Prediction: Are We Making Progress? [84.43405961569256]
  We shed light on the state-of-the-art of network embedding methods for link prediction. We show, using a consistent evaluation pipeline, that only thin progress has been made over the last years. We argue that standardized evaluation tools can repair this situation and boost future progress in this field.
  arXiv  Detail & Related papers  (2020-02-25T16:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.