Diffusion-Free Graph Generation with Next-Scale Prediction

• URL: http://arxiv.org/abs/2503.23612v2
• Date: Thu, 12 Jun 2025 12:54:10 GMT
• Title: Diffusion-Free Graph Generation with Next-Scale Prediction
• Authors: Samuel Belkadi, Steve Hong, Marian Chen, Miruna Cretu, Charles Harris, Pietro Lio,
• Abstract summary: We propose a novel diffusion-free graph generation framework based on next-scale prediction.<n>By leveraging a hierarchy of latent representations, the model progressively generates scales of the entire graph without the need for explicit node ordering.<n>Experiments on both generic and molecular graph datasets demonstrated the potential of this method, achieving inference speedups of up to three orders of magnitude over state-of-the-art methods.
• Score: 3.505533791554976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoregressive models excel in efficiency and plug directly into the transformer ecosystem, delivering robust generalization, predictable scalability, and seamless workflows such as fine-tuning and parallelized training. However, they require an explicit sequence order, which contradicts the unordered nature of graphs. In contrast, diffusion models maintain permutation invariance and enable one-shot generation but require up to thousands of denoising steps and additional features for expressivity, leading to high computational costs. Inspired by recent breakthroughs in image generation, especially the success of visual autoregressive methods, we propose MAG, a novel diffusion-free graph generation framework based on next-scale prediction. By leveraging a hierarchy of latent representations, the model progressively generates scales of the entire graph without the need for explicit node ordering. Experiments on both generic and molecular graph datasets demonstrated the potential of this method, achieving inference speedups of up to three orders of magnitude over state-of-the-art methods, while preserving high-quality generation.

Related papers

• Learning-Order Autoregressive Models with Application to Molecular Graph Generation [52.44913282062524]
  We introduce a variant of ARM that generates high-dimensional data using a probabilistic ordering that is sequentially inferred from data. We demonstrate experimentally that our method can learn meaningful autoregressive orderings in image and graph generation.
  arXiv  Detail & Related papers  (2025-03-07T23:24:24Z)
• Towards Fast Graph Generation via Autoregressive Noisy Filtration Modeling [12.737028324709609]
  Graph generative models often face a critical trade-off between learning complex distributions and achieving fast generation speed.<n>We introduce Autoregressive Noisy filtration Modeling (ANFM), a novel approach that addresses both challenges.<n>ANFM produces remarkably short sequences, achieving a 100-fold speedup in generation time compared to diffusion models.
  arXiv  Detail & Related papers  (2025-02-04T15:35:25Z)
• Flatten Graphs as Sequences: Transformers are Scalable Graph Generators [5.5575224613422725]
  We introduce AutoGraph, a novel framework for generating large attributed graphs using decoder-only transformers.<n>At the core of our approach is a reversible "flattening" process that transforms graphs into random sequences.<n>By sampling and learning from these sequences, AutoGraph enables transformers to model and generate complex graph structures.
  arXiv  Detail & Related papers  (2025-02-04T10:52:14Z)
• IFH: a Diffusion Framework for Flexible Design of Graph Generative Models [53.219279193440734]
  Graph generative models can be classified into two prominent families: one-shot models, which generate a graph in one go, and sequential models, which generate a graph by successive additions of nodes and edges. This paper proposes a graph generative model, called Insert-Fill-Halt (IFH), that supports the specification of a sequentiality degree.
  arXiv  Detail & Related papers  (2024-08-23T16:24:40Z)
• Random Walk Diffusion for Efficient Large-Scale Graph Generation [0.43108040967674194]
  We propose ARROW-Diff (AutoRegressive RandOm Walk Diffusion), a novel random walk-based diffusion approach for efficient large-scale graph generation.<n>We demonstrate that ARROW-Diff can scale to large graphs efficiently, surpassing other baseline methods in terms of both generation time and multiple graph statistics.
  arXiv  Detail & Related papers  (2024-08-08T13:42:18Z)
• Advancing Graph Generation through Beta Diffusion [49.49740940068255]
  Graph Beta Diffusion (GBD) is a generative model specifically designed to handle the diverse nature of graph data. We propose a modulation technique that enhances the realism of generated graphs by stabilizing critical graph topology.
  arXiv  Detail & Related papers  (2024-06-13T17:42:57Z)
• GraphRCG: Self-Conditioned Graph Generation [78.69810678803248]
  We propose a novel self-conditioned graph generation framework designed to explicitly model graph distributions. Our framework demonstrates superior performance over existing state-of-the-art graph generation methods in terms of graph quality and fidelity to training data.
  arXiv  Detail & Related papers  (2024-03-02T02:28:20Z)
• Deep Prompt Tuning for Graph Transformers [55.2480439325792]
  Fine-tuning is resource-intensive and requires storing multiple copies of large models. We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning. By freezing the pre-trained parameters and only updating the added tokens, our approach reduces the number of free parameters and eliminates the need for multiple model copies.
  arXiv  Detail & Related papers  (2023-09-18T20:12:17Z)
• Autoregressive Diffusion Model for Graph Generation [12.390149720274904]
  We propose an emphautoregressive diffusion model for graph generation. Unlike existing methods, we define a node-absorbing diffusion process that operates directly in the discrete graph space. Our experiments on six diverse generic graph datasets and two molecule datasets show that our model achieves better or comparable generation performance with previous state-of-the-art.
  arXiv  Detail & Related papers  (2023-07-17T21:21:18Z)
• Graph Generation with Diffusion Mixture [57.78958552860948]
  Generation of graphs is a major challenge for real-world tasks that require understanding the complex nature of their non-Euclidean structures. We propose a generative framework that models the topology of graphs by explicitly learning the final graph structures of the diffusion process.
  arXiv  Detail & Related papers  (2023-02-07T17:07:46Z)
• Generative Diffusion Models on Graphs: Methods and Applications [50.44334458963234]
  Diffusion models, as a novel generative paradigm, have achieved remarkable success in various image generation tasks. Graph generation is a crucial computational task on graphs with numerous real-world applications.
  arXiv  Detail & Related papers  (2023-02-06T06:58:17Z)
• DiGress: Discrete Denoising diffusion for graph generation [79.13904438217592]
  DiGress is a discrete denoising diffusion model for generating graphs with categorical node and edge attributes. It achieves state-of-the-art performance on molecular and non-molecular datasets, with up to 3x validity improvement. It is also the first model to scale to the large GuacaMol dataset containing 1.3M drug-like molecules.
  arXiv  Detail & Related papers  (2022-09-29T12:55:03Z)
• Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation [18.03898476141173]
  A graph generative model defines a distribution over graphs. We derive the exact joint probability over the graph and the node ordering of the sequential process. We train graph generative models by maximizing this bound, without using the ad-hoc node orderings of previous methods.
  arXiv  Detail & Related papers  (2021-06-11T06:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.