Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks

• URL: http://arxiv.org/abs/2410.09696v1
• Date: Sun, 13 Oct 2024 02:22:14 GMT
• Title: Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks
• Authors: Chaojie Wang, Xinyang Liu, Dongsheng Wang, Hao Zhang, Bo Chen, Mingyuan Zhou,
• Abstract summary: We develop a graph Poisson factor analysis (GPFA) which provides analytic conditional posteriors to improve the inference accuracy. We also extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.
• Score: 50.42343781348247
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although existing variational graph autoencoders (VGAEs) have been widely used for modeling and generating graph-structured data, most of them are still not flexible enough to approximate the sparse and skewed latent node representations, especially those of document relational networks (DRNs) with discrete observations. To analyze a collection of interconnected documents, a typical branch of Bayesian models, specifically relational topic models (RTMs), has proven their efficacy in describing both link structures and document contents of DRNs, which motives us to incorporate RTMs with existing VGAEs to alleviate their potential issues when modeling the generation of DRNs. In this paper, moving beyond the sophisticated approximate assumptions of traditional RTMs, we develop a graph Poisson factor analysis (GPFA), which provides analytic conditional posteriors to improve the inference accuracy, and extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Then, taking GPGBN as the decoder, we combine it with various Weibull-based graph inference networks, resulting in two variants of Weibull graph auto-encoder (WGAE), equipped with model inference algorithms. Experimental results demonstrate that our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.

Related papers

• Signed Graph Autoencoder for Explainable and Polarization-Aware Network Embeddings [20.77134976354226]
  Signed Graph Archetypal Autoencoder (SGAAE) framework designed for signed networks. SGAAE extracts node-level representations that express node memberships over distinct extreme profiles. Model achieves high performance in different tasks of signed link prediction across four real-world datasets.
  arXiv  Detail & Related papers  (2024-09-16T16:40:40Z)
• Network Intrusion Detection with Edge-Directed Graph Multi-Head Attention Networks [13.446986347747325]
  This paper proposes novel Edge-Directed Graph Multi-Head Attention Networks (EDGMAT) for network intrusion detection. The proposed EDGMAT model introduces a multi-head attention mechanism into the intrusion detection model. Additional weight learning is realized through the combination of a multi-head attention mechanism and edge features.
  arXiv  Detail & Related papers  (2023-10-26T12:30:11Z)
• Article Classification with Graph Neural Networks and Multigraphs [0.12499537119440243]
  We propose a method to enhance the performance of article classification by enriching simple Graph Neural Network (GNN) pipelines with multi-graph representations. fully supervised transductive node classification experiments are conducted on the Open Graph Benchmark OGBN-arXiv dataset and the PubMed diabetes dataset. Results demonstrate that multi-graphs consistently improve the performance of a variety of GNN models compared to the default graphs.
  arXiv  Detail & Related papers  (2023-09-20T14:18:04Z)
• Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis [50.972595036856035]
  We present a code that successfully replicates results from six popular and recent graph recommendation models. We compare these graph models with traditional collaborative filtering models that historically performed well in offline evaluations. By investigating the information flow from users' neighborhoods, we aim to identify which models are influenced by intrinsic features in the dataset structure.
  arXiv  Detail & Related papers  (2023-08-01T09:31:44Z)
• GrannGAN: Graph annotation generative adversarial networks [72.66289932625742]
  We consider the problem of modelling high-dimensional distributions and generating new examples of data with complex relational feature structure coherent with a graph skeleton. The model we propose tackles the problem of generating the data features constrained by the specific graph structure of each data point by splitting the task into two phases. In the first it models the distribution of features associated with the nodes of the given graph, in the second it complements the edge features conditionally on the node features.
  arXiv  Detail & Related papers  (2022-12-01T11:49:07Z)
• Text Representation Enrichment Utilizing Graph based Approaches: Stock Market Technical Analysis Case Study [0.0]
  We propose a transductive hybrid approach composed of an unsupervised node representation learning model followed by a node classification/edge prediction model. The proposed model is developed to classify stock market technical analysis reports, which to our knowledge is the first work in this domain.
  arXiv  Detail & Related papers  (2022-11-29T11:26:08Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Benchmarking Graph Neural Networks on Link Prediction [80.2049358846658]
  We benchmark several existing graph neural network (GNN) models on different datasets for link predictions. Our experiments show these GNN architectures perform similarly on various benchmarks for link prediction tasks.
  arXiv  Detail & Related papers  (2021-02-24T20:57:16Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.