Directional diffusion models for graph representation learning

• URL: http://arxiv.org/abs/2306.13210v1
• Date: Thu, 22 Jun 2023 21:27:48 GMT
• Title: Directional diffusion models for graph representation learning
• Authors: Run Yang, Yuling Yang, Fan Zhou, Qiang Sun
• Abstract summary: We propose a new class of models called it directional diffusion models These models incorporate data-dependent, anisotropic, and directional noises in the forward diffusion process. We conduct extensive experiments on 12 publicly available datasets, focusing on two distinct graph representation learning tasks.
• Score: 9.457273750874357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, diffusion models have achieved remarkable success in various domains of artificial intelligence, such as image synthesis, super-resolution, and 3D molecule generation. However, the application of diffusion models in graph learning has received relatively little attention. In this paper, we address this gap by investigating the use of diffusion models for unsupervised graph representation learning. We begin by identifying the anisotropic structures of graphs and a crucial limitation of the vanilla forward diffusion process in learning anisotropic structures. This process relies on continuously adding an isotropic Gaussian noise to the data, which may convert the anisotropic signals to noise too quickly. This rapid conversion hampers the training of denoising neural networks and impedes the acquisition of semantically meaningful representations in the reverse process. To address this challenge, we propose a new class of models called {\it directional diffusion models}. These models incorporate data-dependent, anisotropic, and directional noises in the forward diffusion process. To assess the efficacy of our proposed models, we conduct extensive experiments on 12 publicly available datasets, focusing on two distinct graph representation learning tasks. The experimental results demonstrate the superiority of our models over state-of-the-art baselines, indicating their effectiveness in capturing meaningful graph representations. Our studies not only provide valuable insights into the forward process of diffusion models but also highlight the wide-ranging potential of these models for various graph-related tasks.

Related papers

• Diffusion Models and Representation Learning: A Survey [3.8861148837000856]
  This survey explores the interplay between diffusion models and representation learning. It provides an overview of diffusion models' essential aspects, including mathematical foundations. Various approaches related to diffusion models and representation learning are detailed.
  arXiv  Detail & Related papers  (2024-06-30T17:59:58Z)
• An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization [59.63880337156392]
  Diffusion models have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology. Despite the significant empirical success, theory of diffusion models is very limited. This paper provides a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.
  arXiv  Detail & Related papers  (2024-04-11T14:07:25Z)
• Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
  Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
  arXiv  Detail & Related papers  (2024-02-16T16:47:21Z)
• Diffusion-based Graph Generative Methods [51.04666253001781]
  We systematically and comprehensively review on diffusion-based graph generative methods. We first make a review on three mainstream paradigms of diffusion methods, which are denoising diffusion models, score-based genrative models, and differential equations. In the end, we point out some limitations of current studies and future directions of future explorations.
  arXiv  Detail & Related papers  (2024-01-28T10:09:05Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models [23.70476220346754]
  We propose a novel guidance approach for diffusion models that uses estimated depth information derived from the rich intermediate representations of diffusion models. Experiments and extensive ablation studies demonstrate the effectiveness of our method in guiding the diffusion models toward geometrically plausible image generation.
  arXiv  Detail & Related papers  (2022-12-17T12:47:19Z)
• Fast Graph Generative Model via Spectral Diffusion [38.31052833073743]
  We argue that running full-rank diffusion SDEs on the whole space hinders diffusion models from learning graph topology generation. We propose an efficient yet effective Graph Spectral Diffusion Model (GSDM), which is driven by low-rank diffusion SDEs on the graph spectrum space.
  arXiv  Detail & Related papers  (2022-11-16T12:56:32Z)
• Diffusion Models in Vision: A Survey [80.82832715884597]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)
• Diffusion Models: A Comprehensive Survey of Methods and Applications [10.557289965753437]
  Diffusion models are a class of deep generative models that have shown impressive results on various tasks with dense theoretical founding. Recent studies have shown great enthusiasm on improving the performance of diffusion model.
  arXiv  Detail & Related papers  (2022-09-02T02:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.