Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference

• URL: http://arxiv.org/abs/2605.26552v2
• Date: Wed, 27 May 2026 07:59:16 GMT
• Title: Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference
• Abstract summary: We introduce FAV, Few-step Generative Models Alignment via Sample-based Variational Inference.<n>FAV cast alignment as sampling from a reward-tilted distribution anchored to a reference distribution.<n>We evaluate FAV on two domains: robotics manipulation and image generator alignment.
• Score: 60.82875447631739
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aligning a few-step generative model is challenging, since existing alignment frameworks typically rely on restrictive assumptions: a tractable likelihood, a specific ODE/SDE solver, or a particular model family. We introduce FAV, Few-step Generative Models Alignment via Sample-based Variational Inference, a general alignment framework that requires only sample access to the generator and the reference distribution. We cast alignment as sampling from a reward-tilted distribution anchored to a reference distribution. We leverage Stein Variational Gradient Descent as a sample-based variational inference scheme and amortize its particle updates into the generator parameters via fixed-point regression. We evaluate FAV on two domains: robotics manipulation and image generator alignment. On generative policy alignment for robotic manipulation, FAV outperforms prevailing policy extraction baselines across 56 offline and 30 offline-to-online RL tasks. For image generator alignment, FAV fine-tunes diverse few-step backbones, including GAN, drifting model, consistency models, and flow maps, scaling from ImageNet-$256$ to 1024$^2$ text-to-image synthesis. Code is available at https://github.com/Jaewoopudding/FAV.

Related papers

• Test-Time Weak-to-Strong Alignment: Transferring Implicit Rewards from Weak to Strong Flow Models [19.4335297564654]
  Test-time alignment steers a frozen model during sampling, allowing task-specific and sample-specific guidance.<n>We propose changing the supervision source: let a pair of weak models, not a reward function, supply the supervision.<n>A source aligned model, kept together with its base as a source alignment pair, stores its training reward as an implicit, step-wise, KL-anchored signal.
  arXiv  Detail & Related papers  (2026-09-05T08:12:49Z)
• Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow [10.90140528360878]
  We propose a reward-guided fine-tuning of a one-step generative model via Wasserstein Gradient Flow.<n>We derive a practical training method that requires no reward gradients, thereby handling both non-differentiable and differentiable rewards.<n>Our method provides smooth and stable reward-guided distributional updates while mitigating reward hacking and mode collapse.
  arXiv  Detail & Related papers  (2026-08-30T08:18:30Z)
• Three-Body Scattering for Generative Modeling [17.851305398372794]
  We show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator.<n>Three-Body Scattering Modeling for generation turns the energy distance into a constant-size per-projectile interaction.<n>We provide a design map relating diffusion-related supervision, Drift-like dynamics, and GAN-like objectives.
  arXiv  Detail & Related papers  (2026-07-20T17:38:12Z)
• What Linear Probes Miss: Multi-View Probing for Weight-Space Learning [8.474818775416637]
  MVProbe is a multi-perspective probing framework that synthesizes first-order signals with interaction-aware views.<n>On the Model Jungle benchmark, MVProbe consistently outperforms the state-of-the-art ProbeX.
  arXiv  Detail & Related papers  (2026-05-22T09:18:01Z)
• Fair Benchmarking of Emerging One-Step Generative Models Against Multistep Diffusion and Flow Models [4.809245505572861]
  State-of-the-art text-to-image models produce high-quality images, but inference remains expensive.<n>One-step models aim to reduce this cost by mapping noise to an image in a single step.<n>We show that FID-focused model development and CFG selection can be misleading in few-step regimes.
  arXiv  Detail & Related papers  (2026-03-15T02:22:27Z)
• Generative Modeling with Bayesian Sample Inference [50.07758840675341]
  We derive a novel generative model from iterative Gaussian posterior inference.<n>Our model uses a sequence of prediction and posterior update steps to iteratively narrow down the unknown sample.<n>In experiments, we demonstrate that our model improves sample quality on ImageNet32 over both BFNs and the closely related Variational Diffusion Models.
  arXiv  Detail & Related papers  (2025-02-11T14:27:10Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models [77.47505141269035]
  Generative Visual Prompt (PromptGen) is a framework for distributional control over pre-trained generative models. PromptGen approximats an energy-based model (EBM) and samples images in a feed-forward manner. Code is available at https://github.com/ChenWu98/Generative-Visual-Prompt.
  arXiv  Detail & Related papers  (2022-09-14T22:55:18Z)
• Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting [0.0]
  We introduce a hierarchical latent structure into a VAE-based trajectory forecasting model. Our model is capable of generating clear multi-modal trajectory distributions and outperforms the state-of-the-art (SOTA) models in terms of prediction accuracy.
  arXiv  Detail & Related papers  (2022-07-11T04:52:28Z)
• Self-Conditioned Generative Adversarial Networks for Image Editing [61.50205580051405]
  Generative Adversarial Networks (GANs) are susceptible to bias, learned from either the unbalanced data, or through mode collapse. We argue that this bias is responsible not only for fairness concerns, but that it plays a key role in the collapse of latent-traversal editing methods when deviating away from the distribution's core.
  arXiv  Detail & Related papers  (2022-02-08T18:08:24Z)
• Unsupervised Controllable Generation with Self-Training [90.04287577605723]
  controllable generation with GANs remains a challenging research problem. We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-17T21:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.