Proxy-Free GFlowNet

• URL: http://arxiv.org/abs/2505.20110v1
• Date: Mon, 26 May 2025 15:12:22 GMT
• Title: Proxy-Free GFlowNet
• Authors: Ruishuo Chen, Xun Wang, Rui Hu, Zhuoran Li, Longbo Huang,
• Abstract summary: Generative Flow Networks (GFlowNets) are designed to sample diverse, high-reward structures by modeling distributions over compositional objects.<n>Most existing methods adopt a model-based approach, learning a proxy model from the dataset to approximate the reward function.<n>We propose textbfTrajectory-Distilled GFlowNet (TD-GFN), a emphproxy-free training framework that eliminates the need for out-of-dataset reward queries.
• Score: 39.964801793885485
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Flow Networks (GFlowNets) are a promising class of generative models designed to sample diverse, high-reward structures by modeling distributions over compositional objects. In many real-world applications, obtaining the reward function for such objects is expensive, time-consuming, or requires human input, making it necessary to train GFlowNets from historical datasets. Most existing methods adopt a model-based approach, learning a proxy model from the dataset to approximate the reward function. However, this strategy inherently ties the quality of the learned policy to the accuracy of the proxy, introducing additional complexity and uncertainty into the training process. To overcome these limitations, we propose \textbf{Trajectory-Distilled GFlowNet (TD-GFN)}, a \emph{proxy-free} training framework that eliminates the need for out-of-dataset reward queries. Our method is motivated by the key observation that different edges in the associated directed acyclic graph (DAG) contribute unequally to effective policy learning. TD-GFN leverages inverse reinforcement learning to estimate edge-level rewards from the offline dataset, which are then used to ingeniously prune the DAG and guide backward trajectory sampling during training. This approach directs the policy toward high-reward regions while reducing the complexity of model fitting. Empirical results across multiple tasks show that TD-GFN trains both efficiently and reliably, significantly outperforming existing baselines in convergence speed and sample quality.

Related papers

• A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation [16.82426251068573]
  Link Prediction (LP) is a critical task in graph machine learning.<n>Existing methods face key challenges including limited supervision from sparse connectivity.<n>We explore pretraining as a solution to address these challenges.
  arXiv  Detail & Related papers  (2025-08-06T17:10:31Z)
• Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling [53.11315884128402]
  Temporal Knowledge Graph (TKG) completion models traditionally assume access to the entire graph during training.<n>We present an incremental training framework specifically designed for TKGs, aiming to address entities that are either not observed during training or have sparse connections.<n>Our approach combines a model-agnostic enhancement layer with a weighted sampling strategy, that can be augmented to and improve any existing TKG completion method.
  arXiv  Detail & Related papers  (2025-07-25T06:02:48Z)
• Fake it till You Make it: Reward Modeling as Discriminative Prediction [49.31309674007382]
  GAN-RM is an efficient reward modeling framework that eliminates manual preference annotation and explicit quality dimension engineering.<n>Our method trains the reward model through discrimination between a small set of representative, unpaired target samples.<n>Experiments demonstrate our GAN-RM's effectiveness across multiple key applications.
  arXiv  Detail & Related papers  (2025-06-16T17:59:40Z)
• Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners [82.72552644267724]
  BoostPFN can outperform standard PFNs with the same size of training samples in large datasets.<n>High performance is maintained for up to 50x of the pre-training size of PFNs.
  arXiv  Detail & Related papers  (2025-03-03T07:31:40Z)
• Subsampling Graphs with GNN Performance Guarantees [34.32848091746629]
  We introduce new subsampling methods for graph datasets.<n>We prove that training a GNN on the subsampled data results in a bounded increase in loss compared to training on the full dataset.
  arXiv  Detail & Related papers  (2025-02-23T20:21:16Z)
• Efficient Training of Deep Neural Operator Networks via Randomized Sampling [0.0]
  Deep operator network (DeepNet) has demonstrated success in the real-time prediction of complex dynamics across various scientific and engineering applications. We introduce a random sampling technique to be adopted the training of DeepONet, aimed at improving generalization ability of the model, while significantly reducing computational time. Our results indicate that incorporating randomization in the trunk network inputs during training enhances the efficiency and robustness of DeepONet, offering a promising avenue for improving the framework's performance in modeling complex physical systems.
  arXiv  Detail & Related papers  (2024-09-20T07:18:31Z)
• Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets [27.33222647437964]
  Generative Flow Networks (GFlowNets) have demonstrated remarkable capabilities to generate diverse sets of high-reward candidates.<n>However, training such models is challenging due to extremely sparse rewards.<n>We propose a novel method called textbfRetrospective textbfBackward textbfSynthesis (textbfRBS) to address these problems.
  arXiv  Detail & Related papers  (2024-06-03T09:44:10Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Pre-Training and Fine-Tuning Generative Flow Networks [61.90529626590415]
  We introduce a novel approach for reward-free pre-training of GFlowNets. By framing the training as a self-supervised problem, we propose an outcome-conditioned GFlowNet that learns to explore the candidate space. We show that the pre-trained OC-GFN model can allow for a direct extraction of a policy capable of sampling from any new reward functions in downstream tasks.
  arXiv  Detail & Related papers  (2023-10-05T09:53:22Z)
• Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphs [34.76241250013461]
  We formulate the graph active learning problem as a generative process, named GFlowGNN, which generates various samples through sequential actions. We show that the proposed approach has good exploration capability and transferability, outperforming various state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-24T10:47:08Z)
• Analyzing the Effect of Sampling in GNNs on Individual Fairness [79.28449844690566]
  Graph neural network (GNN) based methods have saturated the field of recommender systems. We extend an existing method for promoting individual fairness on graphs to support mini-batch, or sub-sample based, training of a GNN. We show that mini-batch training facilitate individual fairness promotion by allowing for local nuance to guide the process of fairness promotion in representation learning.
  arXiv  Detail & Related papers  (2022-09-08T16:20:25Z)
• Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks [52.566735716983956]
  We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs. Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs. Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
  arXiv  Detail & Related papers  (2022-07-18T14:23:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.