Consistent Training via Energy-Based GFlowNets for Modeling Discrete Joint Distributions

• URL: http://arxiv.org/abs/2211.00568v2
• Date: Wed, 2 Nov 2022 13:12:50 GMT
• Title: Consistent Training via Energy-Based GFlowNets for Modeling Discrete Joint Distributions
• Authors: Chanakya Ekbote, Moksh Jain, Payel Das, Yoshua Bengio
• Abstract summary: Generative Flow Networks (GFlowNets) have demonstrated significant performance improvements for generating diverse objects $x$ given a reward function $R(x)$. We build upon recent work on jointly learning energy-based models with GFlowNets and extend it to learn the joint over multiple variables. We find that this joint training or joint energy-based formulation leads to significant improvements in generating anti-microbial peptides.
• Score: 79.47120571256026
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Flow Networks (GFlowNets) have demonstrated significant performance improvements for generating diverse discrete objects $x$ given a reward function $R(x)$, indicating the utility of the object and trained independently from the GFlowNet by supervised learning to predict a desirable property $y$ given $x$. We hypothesize that this can lead to incompatibility between the inductive optimization biases in training $R$ and in training the GFlowNet, potentially leading to worse samples and slow adaptation to changes in the distribution. In this work, we build upon recent work on jointly learning energy-based models with GFlowNets and extend it to learn the joint over multiple variables, which we call Joint Energy-Based GFlowNets (JEBGFNs), such as peptide sequences and their antimicrobial activity. Joint learning of the energy-based model, used as a reward for the GFlowNet, can resolve the issues of incompatibility since both the reward function $R$ and the GFlowNet sampler are trained jointly. We find that this joint training or joint energy-based formulation leads to significant improvements in generating anti-microbial peptides. As the training sequences arose out of evolutionary or artificial selection for high antibiotic activity, there is presumably some structure in the distribution of sequences that reveals information about the antibiotic activity. This results in an advantage to modeling their joint generatively vs. pure discriminative modeling. We also evaluate JEBGFN in an active learning setting for discovering anti-microbial peptides.

Related papers

• Improving GFlowNets with Monte Carlo Tree Search [6.497027864860203]
  Recent studies have revealed strong connections between GFlowNets and entropy-regularized reinforcement learning. We propose to enhance planning capabilities of GFlowNets by applying Monte Carlo Tree Search (MCTS) Our experiments demonstrate that this approach improves the sample efficiency of GFlowNet training and the generation fidelity of pre-trained GFlowNet models.
  arXiv  Detail & Related papers  (2024-06-19T15:58:35Z)
• Baking Symmetry into GFlowNets [58.932776403471635]
  GFlowNets have exhibited promising performance in generating diverse candidates with high rewards. This study aims to integrate symmetries into GFlowNets by identifying equivalent actions during the generation process.
  arXiv  Detail & Related papers  (2024-06-08T10:11:10Z)
• Rectifying Reinforcement Learning for Reward Matching [12.294107455811496]
  We establish a new connection between GFlowNets and policy evaluation for a uniform policy. We propose a novel rectified policy evaluation algorithm, which achieves the same reward-matching effect as GFlowNets.
  arXiv  Detail & Related papers  (2024-06-04T11:11:53Z)
• Evolution Guided Generative Flow Networks [11.609895436955242]
  Generative Flow Networks (GFlowNets) learn to sample compositional objects proportional to their rewards. One big challenge of GFlowNets is training them effectively when dealing with long time horizons and sparse rewards. We propose Evolution guided generative flow networks (EGFN), a simple but powerful augmentation to the GFlowNets training using Evolutionary algorithms (EA)
  arXiv  Detail & Related papers  (2024-02-03T15:28:53Z)
• Generative Augmented Flow Networks [88.50647244459009]
  We propose Generative Augmented Flow Networks (GAFlowNets) to incorporate intermediate rewards into GFlowNets. GAFlowNets can leverage edge-based and state-based intrinsic rewards in a joint way to improve exploration.
  arXiv  Detail & Related papers  (2022-10-07T03:33:56Z)
• Learning GFlowNets from partial episodes for improved convergence and stability [56.99229746004125]
  Generative flow networks (GFlowNets) are algorithms for training a sequential sampler of discrete objects under an unnormalized target density. Existing training objectives for GFlowNets are either local to states or transitions, or propagate a reward signal over an entire sampling trajectory. Inspired by the TD($lambda$) algorithm in reinforcement learning, we introduce subtrajectory balance or SubTB($lambda$), a GFlowNet training objective that can learn from partial action subsequences of varying lengths.
  arXiv  Detail & Related papers  (2022-09-26T15:44:24Z)
• Generative Flow Networks for Discrete Probabilistic Modeling [118.81967600750428]
  We present energy-based generative flow networks (EB-GFN) EB-GFN is a novel probabilistic modeling algorithm for high-dimensional discrete data. We show how GFlowNets can approximately perform large-block Gibbs sampling to mix between modes.
  arXiv  Detail & Related papers  (2022-02-03T01:27:11Z)
• GFlowNet Foundations [66.69854262276391]
  Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context. We show a number of additional theoretical properties of GFlowNets.
  arXiv  Detail & Related papers  (2021-11-17T17:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.