Disentangled Condensation for Large-scale Graphs

• URL: http://arxiv.org/abs/2401.12231v2
• Date: Wed, 31 Jul 2024 02:52:20 GMT
• Title: Disentangled Condensation for Large-scale Graphs
• Authors: Zhenbang Xiao, Shunyu Liu, Yu Wang, Tongya Zheng, Mingli Song,
• Abstract summary: Graph condensation has emerged as an intriguing technique to save the expensive training costs of Graph Neural Networks (GNNs) We propose to disentangle the condensation process into a two-stage GNN-free paradigm, independently condensing nodes and generating edges. This simple yet effective approach achieves at least 10 times faster than state-of-the-art methods with comparable accuracy on medium-scale graphs.
• Score: 31.781721873508978
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph condensation has emerged as an intriguing technique to save the expensive training costs of Graph Neural Networks (GNNs) by substituting a condensed small graph with the original graph. Despite the promising results achieved, previous methods usually employ an entangled paradigm of redundant parameters (nodes, edges, GNNs), which incurs complex joint optimization during condensation. This paradigm has considerably impeded the scalability of graph condensation, making it challenging to condense extremely large-scale graphs and generate high-fidelity condensed graphs. Therefore, we propose to disentangle the condensation process into a two-stage GNN-free paradigm, independently condensing nodes and generating edges while eliminating the need to optimize GNNs at the same time. The node condensation module avoids the complexity of GNNs by focusing on node feature alignment with anchors of the original graph, while the edge translation module constructs the edges of the condensed nodes by transferring the original structure knowledge with neighborhood anchors. This simple yet effective approach achieves at least 10 times faster than state-of-the-art methods with comparable accuracy on medium-scale graphs. Moreover, the proposed DisCo can successfully scale up to the Ogbn-papers100M graph with flexible reduction rates. Extensive downstream tasks and ablation study on five common datasets further demonstrate the effectiveness of the proposed DisCo framework. The source code will be made publicly available.

Related papers

• Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning [47.74244053386216]
  Graph condensation is a promising solution to synthesize a compact, substitute graph of the large-scale original graph. We introduce Contrastive Graph Condensation (CTGC), which adopts a self-supervised surrogate task to extract critical, causal information from the original graph. CTGC excels in handling various downstream tasks with a limited number of labels, consistently outperforming state-of-the-art GC methods.
  arXiv  Detail & Related papers  (2024-11-26T03:01:22Z)
• Graph Condensation for Open-World Graph Learning [48.38802327346445]
  Graph condensation (GC) has emerged as a promising acceleration solution for efficiently training graph neural networks (GNNs) Existing GC methods are limited to aligning the condensed graph with merely the observed static graph distribution. In real-world scenarios, however, graphs are dynamic and constantly evolving, with new nodes and edges being continually integrated. We propose OpenGC, a robust GC framework that integrates structure-aware distribution shift to simulate evolving graph patterns.
  arXiv  Detail & Related papers  (2024-05-27T09:47:09Z)
• Rethinking and Accelerating Graph Condensation: A Training-Free Approach with Class Partition [56.26113670151363]
  Graph condensation is a data-centric solution to replace the large graph with a small yet informative condensed graph. Existing GC methods suffer from intricate optimization processes, necessitating excessive computing resources. We propose a training-free GC framework termed Class-partitioned Graph Condensation (CGC) CGC achieves state-of-the-art performance with a more efficient condensation process.
  arXiv  Detail & Related papers  (2024-05-22T14:57:09Z)
• Simple Graph Condensation [30.85754566420301]
  Graph condensation involves tuning Graph Neural Networks (GNNs) on a small condensed graph for use on a large-scale original graph. We introduce the Simple Graph Condensation (SimGC) framework, which aligns the condensed graph with the original graph from the input layer to the prediction layer. SimGC achieves a significant speedup of up to 10 times compared to existing graph condensation methods.
  arXiv  Detail & Related papers  (2024-03-22T05:04:48Z)
• Graph Condensation for Inductive Node Representation Learning [59.76374128436873]
  We propose mapping-aware graph condensation (MCond) MCond integrates new nodes into the synthetic graph for inductive representation learning. On the Reddit dataset, MCond achieves up to 121.5x inference speedup and 55.9x reduction in storage requirements.
  arXiv  Detail & Related papers  (2023-07-29T12:11:14Z)
• Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data [91.27527985415007]
  Existing graph condensation methods rely on the joint optimization of nodes and structures in the condensed graph. We advocate a new Structure-Free Graph Condensation paradigm, named SFGC, to distill a large-scale graph into a small-scale graph node set.
  arXiv  Detail & Related papers  (2023-06-05T07:53:52Z)
• Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks [52.566735716983956]
  We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs. Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs. Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
  arXiv  Detail & Related papers  (2022-07-18T14:23:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.