Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation

• URL: http://arxiv.org/abs/2402.03687v2
• Date: Thu, 23 May 2024 04:38:47 GMT
• Title: Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation
• Authors: Lingxiao Zhao, Xueying Ding, Leman Akoglu,
• Abstract summary: We introduce a Permutation-invariant Auto Regressive Diffusion model that integrates diffusion models with autoregressive methods. Pard achieves state-of-the-art performance on molecular and non-molecular datasets, and scales to large datasets like MOSES containing 1.9M molecules.
• Score: 36.896680500652536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph generation has been dominated by autoregressive models due to their simplicity and effectiveness, despite their sensitivity to ordering. Yet diffusion models have garnered increasing attention, as they offer comparable performance while being permutation-invariant. Current graph diffusion models generate graphs in a one-shot fashion, but they require extra features and thousands of denoising steps to achieve optimal performance. We introduce PARD, a Permutation-invariant Auto Regressive Diffusion model that integrates diffusion models with autoregressive methods. PARD harnesses the effectiveness and efficiency of the autoregressive model while maintaining permutation invariance without ordering sensitivity. Specifically, we show that contrary to sets, elements in a graph are not entirely unordered and there is a unique partial order for nodes and edges. With this partial order, PARD generates a graph in a block-by-block, autoregressive fashion, where each block's probability is conditionally modeled by a shared diffusion model with an equivariant network. To ensure efficiency while being expressive, we further propose a higher-order graph transformer, which integrates transformer with PPGN. Like GPT, we extend the higher-order graph transformer to support parallel training of all blocks. Without any extra features, PARD achieves state-of-the-art performance on molecular and non-molecular datasets, and scales to large datasets like MOSES containing 1.9M molecules. Pard is open-sourced at https://github.com/LingxiaoShawn/Pard.

Related papers

• IFH: a Diffusion Framework for Flexible Design of Graph Generative Models [53.219279193440734]
  Graph generative models can be classified into two prominent families: one-shot models, which generate a graph in one go, and sequential models, which generate a graph by successive additions of nodes and edges. This paper proposes a graph generative model, called Insert-Fill-Halt (IFH), that supports the specification of a sequentiality degree.
  arXiv  Detail & Related papers  (2024-08-23T16:24:40Z)
• Advancing Graph Generation through Beta Diffusion [49.49740940068255]
  Graph Beta Diffusion (GBD) is a generative model specifically designed to handle the diverse nature of graph data. We propose a modulation technique that enhances the realism of generated graphs by stabilizing critical graph topology.
  arXiv  Detail & Related papers  (2024-06-13T17:42:57Z)
• Autoregressive Diffusion Model for Graph Generation [12.390149720274904]
  We propose an emphautoregressive diffusion model for graph generation. Unlike existing methods, we define a node-absorbing diffusion process that operates directly in the discrete graph space. Our experiments on six diverse generic graph datasets and two molecule datasets show that our model achieves better or comparable generation performance with previous state-of-the-art.
  arXiv  Detail & Related papers  (2023-07-17T21:21:18Z)
• SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation [15.977241867213516]
  Diffusion models based on permutation-equivariant networks can learn permutation-invariant distributions for graph data. We propose a non-invariant diffusion model, called $textitSwinGNN$, which employs an efficient edge-to-edge 2-WL message passing network.
  arXiv  Detail & Related papers  (2023-07-04T10:58:42Z)
• Discrete Graph Auto-Encoder [52.50288418639075]
  We introduce a new framework named Discrete Graph Auto-Encoder (DGAE) We first use a permutation-equivariant auto-encoder to convert graphs into sets of discrete latent node representations. In the second step, we sort the sets of discrete latent representations and learn their distribution with a specifically designed auto-regressive model.
  arXiv  Detail & Related papers  (2023-06-13T12:40:39Z)
• DiGress: Discrete Denoising diffusion for graph generation [79.13904438217592]
  DiGress is a discrete denoising diffusion model for generating graphs with categorical node and edge attributes. It achieves state-of-the-art performance on molecular and non-molecular datasets, with up to 3x validity improvement. It is also the first model to scale to the large GuacaMol dataset containing 1.3M drug-like molecules.
  arXiv  Detail & Related papers  (2022-09-29T12:55:03Z)
• Permutation Invariant Graph Generation via Score-Based Generative Modeling [114.12935776726606]
  We propose a permutation invariant approach to modeling graphs, using the recent framework of score-based generative modeling. In particular, we design a permutation equivariant, multi-channel graph neural network to model the gradient of the data distribution at the input graph. For graph generation, we find that our learning approach achieves better or comparable results to existing models on benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-02T03:06:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.