Variational Flow Matching for Graph Generation

• URL: http://arxiv.org/abs/2406.04843v1
• Date: Fri, 7 Jun 2024 11:16:17 GMT
• Title: Variational Flow Matching for Graph Generation
• Authors: Floor Eijkelboom, Grigory Bartosh, Christian Andersson Naesseth, Max Welling, Jan-Willem van de Meent,
• Abstract summary: We develop CatFlow, a flow matching method for categorical data. CatFlow is easy to implement, computationally efficient, and achieves strong results on graph generation tasks.
• Score: 42.3778673162256
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a formulation of flow matching as variational inference, which we refer to as variational flow matching (VFM). Based on this formulation we develop CatFlow, a flow matching method for categorical data. CatFlow is easy to implement, computationally efficient, and achieves strong results on graph generation tasks. In VFM, the objective is to approximate the posterior probability path, which is a distribution over possible end points of a trajectory. We show that VFM admits both the CatFlow objective and the original flow matching objective as special cases. We also relate VFM to score-based models, in which the dynamics are stochastic rather than deterministic, and derive a bound on the model likelihood based on a reweighted VFM objective. We evaluate CatFlow on one abstract graph generation task and two molecular generation tasks. In all cases, CatFlow exceeds or matches performance of the current state-of-the-art models.

Related papers

• Normalizing Flows are Capable Generative Models [48.31226028595099]
  TarFlow is a simple and scalable architecture that enables highly performant NF models. It is straightforward to train end-to-end, and capable of directly modeling and generating pixels. TarFlow sets new state-of-the-art results on likelihood estimation for images, beating the previous best methods by a large margin.
  arXiv  Detail & Related papers  (2024-12-09T09:28:06Z)
• Stream-level flow matching with Gaussian processes [4.935875591615496]
  Conditional flow matching (CFM) is a family of training algorithms for fitting continuous normalizing flows (CNFs) We extend the CFM algorithm by defining conditional probability paths along streams'', instances of latent paths that connect data pairs of source and target. We show that this generalization of the CFM can effectively reduce the variance in the estimated marginal vector field at a moderate computational cost.
  arXiv  Detail & Related papers  (2024-09-30T15:47:22Z)
• Consistency Flow Matching: Defining Straight Flows with Velocity Consistency [97.28511135503176]
  We introduce Consistency Flow Matching (Consistency-FM), a novel FM method that explicitly enforces self-consistency in the velocity field. Preliminary experiments demonstrate that our Consistency-FM significantly improves training efficiency by converging 4.4x faster than consistency models.
  arXiv  Detail & Related papers  (2024-07-02T16:15:37Z)
• Fisher Flow Matching for Generative Modeling over Discrete Data [12.69975914345141]
  We introduce Fisher-Flow, a novel flow-matching model for discrete data. Fisher-Flow takes a manifestly geometric perspective by considering categorical distributions over discrete data. We prove that the gradient flow induced by Fisher-Flow is optimal in reducing the forward KL divergence.
  arXiv  Detail & Related papers  (2024-05-23T15:02:11Z)
• PaddingFlow: Improving Normalizing Flows with Padding-Dimensional Noise [4.762593660623934]
  We propose PaddingFlow, a novel dequantization method, which improves normalizing flows with padding-dimensional noise. We validate our method on the main benchmarks of unconditional density estimation. The results show that PaddingFlow can perform better in all experiments in this paper.
  arXiv  Detail & Related papers  (2024-03-13T03:28:39Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• Bayesian Structure Learning with Generative Flow Networks [85.84396514570373]
  In Bayesian structure learning, we are interested in inferring a distribution over the directed acyclic graph (DAG) from data. Recently, a class of probabilistic models, called Generative Flow Networks (GFlowNets), have been introduced as a general framework for generative modeling. We show that our approach, called DAG-GFlowNet, provides an accurate approximation of the posterior over DAGs.
  arXiv  Detail & Related papers  (2022-02-28T15:53:10Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds [15.476426879806134]
  Flow-based generative models are composed of invertible transformations between two random variables of the same dimension. In this paper, we propose SoftFlow, a probabilistic framework for training normalizing flows on manifold. We experimentally show that SoftFlow can capture the innate structure of the manifold data and generate high-quality samples. We apply the proposed framework to 3D point clouds to alleviate the difficulty of forming thin structures for flow-based models.
  arXiv  Detail & Related papers  (2020-06-08T13:56:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.