Does Graph Distillation See Like Vision Dataset Counterpart?

• URL: http://arxiv.org/abs/2310.09192v1
• Date: Fri, 13 Oct 2023 15:36:48 GMT
• Title: Does Graph Distillation See Like Vision Dataset Counterpart?
• Authors: Beining Yang, Kai Wang, Qingyun Sun, Cheng Ji, Xingcheng Fu, Hao Tang, Yang You, Jianxin Li
• Abstract summary: We propose a novel structure-broadcasting Graph dataset Distillation (SGDD) scheme for broadcasting the original structure information to the generation of the synthetic one. We validate the proposed SGDD across 9 datasets and achieve state-of-the-art results on all of them.
• Score: 26.530765707382457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have attracted increasing concerns. Existing graph condensation methods primarily focus on optimizing the feature matrices of condensed graphs while overlooking the impact of the structure information from the original graphs. To investigate the impact of the structure information, we conduct analysis from the spectral domain and empirically identify substantial Laplacian Energy Distribution (LED) shifts in previous works. Such shifts lead to poor performance in cross-architecture generalization and specific tasks, including anomaly detection and link prediction. In this paper, we propose a novel Structure-broadcasting Graph Dataset Distillation (SGDD) scheme for broadcasting the original structure information to the generation of the synthetic one, which explicitly prevents overlooking the original structure information. Theoretically, the synthetic graphs by SGDD are expected to have smaller LED shifts than previous works, leading to superior performance in both cross-architecture settings and specific tasks. We validate the proposed SGDD across 9 datasets and achieve state-of-the-art results on all of them: for example, on the YelpChi dataset, our approach maintains 98.6% test accuracy of training on the original graph dataset with 1,000 times saving on the scale of the graph. Moreover, we empirically evaluate there exist 17.6% ~ 31.4% reductions in LED shift crossing 9 datasets. Extensive experiments and analysis verify the effectiveness and necessity of the proposed designs. The code is available in the GitHub repository: https://github.com/RingBDStack/SGDD.

Related papers

• GSTAM: Efficient Graph Distillation with Structural Attention-Matching [13.673737442696154]
  We introduce Graph Distillation with Structural Attention Matching ( GSTAM), a novel method for condensing graph classification datasets. GSTAM leverages the attention maps of GNNs to distill structural information from the original dataset into synthetic graphs. Comprehensive experiments demonstrate GSTAM's superiority over existing methods, achieving 0.45% to 6.5% better performance in extreme condensation ratios.
  arXiv  Detail & Related papers  (2024-08-29T19:40:04Z)
• Greener GRASS: Enhancing GNNs with Encoding, Rewiring, and Attention [12.409982249220812]
  We introduce Graph Attention with Structures (GRASS), a novel GNN architecture, to enhance graph relative attention. GRASS rewires the input graph by superimposing a random regular graph to achieve long-range information propagation. It also employs a novel additive attention mechanism tailored for graph-structured data.
  arXiv  Detail & Related papers  (2024-07-08T06:21:56Z)
• Amplify Graph Learning for Recommendation via Sparsity Completion [16.32861024767423]
  Graph learning models have been widely deployed in collaborative filtering (CF) based recommendation systems. Due to the issue of data sparsity, the graph structure of the original input lacks potential positive preference edges. We propose an Amplify Graph Learning framework based on Sparsity Completion (called AGL-SC)
  arXiv  Detail & Related papers  (2024-06-27T08:26:20Z)
• Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching [26.303436980548174]
  Graph condensation aims to reduce the size of a large-scale graph dataset by synthesizing a compact counterpart. Existing methods often fall short of accurately replicating the original graph for certain datasets. In this paper, we make the first attempt toward textitlossless graph condensation by bridging the previously neglected supervision signals.
  arXiv  Detail & Related papers  (2024-02-07T16:32:02Z)
• Deep Manifold Graph Auto-Encoder for Attributed Graph Embedding [51.75091298017941]
  This paper proposes a novel Deep Manifold (Variational) Graph Auto-Encoder (DMVGAE/DMGAE) for attributed graph data. The proposed method surpasses state-of-the-art baseline algorithms by a significant margin on different downstream tasks across popular datasets.
  arXiv  Detail & Related papers  (2024-01-12T17:57:07Z)
• Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis [50.972595036856035]
  We present a code that successfully replicates results from six popular and recent graph recommendation models. We compare these graph models with traditional collaborative filtering models that historically performed well in offline evaluations. By investigating the information flow from users' neighborhoods, we aim to identify which models are influenced by intrinsic features in the dataset structure.
  arXiv  Detail & Related papers  (2023-08-01T09:31:44Z)
• ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning [34.23920789327245]
  We propose ENGAGE, where explanation guides the contrastive augmentation process to preserve the key parts in graphs. We also design two data augmentation schemes on graphs for perturbing structural and feature information, respectively.
  arXiv  Detail & Related papers  (2023-07-03T14:33:14Z)
• Spectral Augmentations for Graph Contrastive Learning [50.149996923976836]
  Contrastive learning has emerged as a premier method for learning representations with or without supervision. Recent studies have shown its utility in graph representation learning for pre-training. We propose a set of well-motivated graph transformation operations to provide a bank of candidates when constructing augmentations for a graph contrastive objective.
  arXiv  Detail & Related papers  (2023-02-06T16:26:29Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)
• Robust Optimization as Data Augmentation for Large-scale Graphs [117.2376815614148]
  We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks.
  arXiv  Detail & Related papers  (2020-10-19T21:51:47Z)
• Heuristic Semi-Supervised Learning for Graph Generation Inspired by Electoral College [80.67842220664231]
  We propose a novel pre-processing technique, namely ELectoral COllege (ELCO), which automatically expands new nodes and edges to refine the label similarity within a dense subgraph. In all setups tested, our method boosts the average score of base models by a large margin of 4.7 points, as well as consistently outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-10T14:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.