From Moments to Models: Graphon Mixture-Aware Mixup and Contrastive Learning

• URL: http://arxiv.org/abs/2510.03690v2
• Date: Thu, 09 Oct 2025 17:55:28 GMT
• Title: From Moments to Models: Graphon Mixture-Aware Mixup and Contrastive Learning
• Authors: Ali Azizpour, Reza Ramezanpour, Ashutosh Sabharwal, Santiago Segarra,
• Abstract summary: We propose a unified framework that explicitly models data as a mixture of underlying graph generative models represented by graphons.<n>This enables us to disentangle the mixture components and identify their distinct generative mechanisms.<n>In unsupervised learning, MGCL achieves state-of-the-art results, obtaining the top average rank across eight datasets.<n>In supervised learning, GMAM consistently outperforms existing strategies, achieving new state-of-the-art accuracy in 6 out of 7 datasets.
• Score: 37.00052626796266
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world graph datasets often consist of mixtures of populations, where graphs are generated from multiple distinct underlying distributions. However, modern representation learning approaches, such as graph contrastive learning (GCL) and augmentation methods like Mixup, typically overlook this mixture structure. In this work, we propose a unified framework that explicitly models data as a mixture of underlying probabilistic graph generative models represented by graphons. To characterize these graphons, we leverage graph moments (motif densities) to cluster graphs arising from the same model. This enables us to disentangle the mixture components and identify their distinct generative mechanisms. This model-aware partitioning benefits two key graph learning tasks: 1) It enables a graphon-mixture-aware mixup (GMAM), a data augmentation technique that interpolates in a semantically valid space guided by the estimated graphons, instead of assuming a single graphon per class. 2) For GCL, it enables model-adaptive and principled augmentations. Additionally, by introducing a new model-aware objective, our proposed approach (termed MGCL) improves negative sampling by restricting negatives to graphs from other models. We establish a key theoretical guarantee: a novel, tighter bound showing that graphs sampled from graphons with small cut distance will have similar motif densities with high probability. Extensive experiments on benchmark datasets demonstrate strong empirical performance. In unsupervised learning, MGCL achieves state-of-the-art results, obtaining the top average rank across eight datasets. In supervised learning, GMAM consistently outperforms existing strategies, achieving new state-of-the-art accuracy in 6 out of 7 datasets.

Related papers

• Beyond MMD: Evaluating Graph Generative Models with Geometric Deep Learning [0.0]
  Graph Generative Models (GGMs) have emerged as a promising solution to the problem of generating realistic graphs.<n>This paper introduces a novel methodology for evaluating GGMs that overcomes the limitations of Maximum Mean Discrepancy (MMD)<n>We present a comprehensive evaluation of two state-of-the-art Graph Generative Models: Graph Recurrent Attention Networks (GRAN) and Efficient and Degree-guided graph GEnerative model (EDGE)
  arXiv  Detail & Related papers  (2025-12-16T09:51:44Z)
• Model-Driven Graph Contrastive Learning [25.015678499211404]
  We propose $textbfMGCL$, a model-driven graph contrastive learning (GCL) framework.<n>GCL has emerged as a powerful self-supervised framework for learning expressive node or graph representations.<n>Experiments on benchmark datasets demonstrate that MGCL achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-06-06T16:17:22Z)
• M3C: A Framework towards Convergent, Flexible, and Unsupervised Learning of Mixture Graph Matching and Clustering [57.947071423091415]
  We introduce Minorize-Maximization Matching and Clustering (M3C), a learning-free algorithm that guarantees theoretical convergence. We develop UM3C, an unsupervised model that incorporates novel edge-wise affinity learning and pseudo label selection. Our method outperforms state-of-the-art graph matching and mixture graph matching and clustering approaches in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-10-27T19:40:34Z)
• Permutation Equivariant Graph Framelets for Heterophilous Graph Learning [6.679929638714752]
  We develop a new way to implement multi-scale extraction via constructing Haar-type graph framelets. We show that our model can achieve the best performance on certain datasets of heterophilous graphs.
  arXiv  Detail & Related papers  (2023-06-07T09:05:56Z)
• Graph Generation with Diffusion Mixture [57.78958552860948]
  Generation of graphs is a major challenge for real-world tasks that require understanding the complex nature of their non-Euclidean structures. We propose a generative framework that models the topology of graphs by explicitly learning the final graph structures of the diffusion process.
  arXiv  Detail & Related papers  (2023-02-07T17:07:46Z)
• Micro and Macro Level Graph Modeling for Graph Variational Auto-Encoders [16.302222204710276]
  This paper proposes a new multi-level framework that jointly models node-level properties and graph-level statistics. We introduce a new micro-macro training objective for graph generation that combines node-level and graph-level losses. Our experiments show that adding micro-macro modeling to the GraphVAE model improves graph quality scores up to 2 orders of magnitude on five benchmark datasets.
  arXiv  Detail & Related papers  (2022-10-30T13:45:21Z)
• Similarity-aware Positive Instance Sampling for Graph Contrastive Pre-training [82.68805025636165]
  We propose to select positive graph instances directly from existing graphs in the training set. Our selection is based on certain domain-specific pair-wise similarity measurements. Besides, we develop an adaptive node-level pre-training method to dynamically mask nodes to distribute them evenly in the graph.
  arXiv  Detail & Related papers  (2022-06-23T20:12:51Z)
• G-Mixup: Graph Data Augmentation for Graph Classification [55.63157775049443]
  Mixup has shown superiority in improving the generalization and robustness of neural networks by interpolating features and labels between two random samples. We propose $mathcalG$-Mixup to augment graphs for graph classification by interpolating the generator (i.e., graphon) of different classes of graphs. Experiments show that $mathcalG$-Mixup substantially improves the generalization and robustness of GNNs.
  arXiv  Detail & Related papers  (2022-02-15T04:09:44Z)
• Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming [48.99614465020678]
  We introduce a novel self-supervised graph representation learning algorithm via Graph Contrastive Adjusted Zooming. This mechanism enables G-Zoom to explore and extract self-supervision signals from a graph from multiple scales. We have conducted extensive experiments on real-world datasets, and the results demonstrate that our proposed model outperforms state-of-the-art methods consistently.
  arXiv  Detail & Related papers  (2021-11-20T22:45:53Z)
• Stochastic Iterative Graph Matching [11.128153575173213]
  We propose a new model, Iterative Graph MAtching, to address the graph matching problem. Our model defines a distribution of matchings for a graph pair so the model can explore a wide range of possible matchings. We conduct extensive experiments across synthetic graph datasets as well as biochemistry and computer vision applications.
  arXiv  Detail & Related papers  (2021-06-04T02:05:35Z)
• Adaptive Graph Auto-Encoder for General Data Clustering [90.8576971748142]
  Graph-based clustering plays an important role in the clustering area. Recent studies about graph convolution neural networks have achieved impressive success on graph type data. We propose a graph auto-encoder for general data clustering, which constructs the graph adaptively according to the generative perspective of graphs.
  arXiv  Detail & Related papers  (2020-02-20T10:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.