Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps

• URL: http://arxiv.org/abs/2602.05993v1
• Date: Thu, 05 Feb 2026 18:42:00 GMT
• Title: Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps
• Abstract summary: Flow and diffusion models produce high-quality samples, but adapting them to user preferences or constraints post-training remains costly and brittle.<n>We argue that efficient reward alignment should be a property of the generative model itself, not an afterthought, and the model for adaptability.<n>We propose "Diamond Maps", flow map models that enable efficient and accurate alignment to arbitrary rewards at inference time.
• Score: 54.18537904256945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Flow and diffusion models produce high-quality samples, but adapting them to user preferences or constraints post-training remains costly and brittle, a challenge commonly called reward alignment. We argue that efficient reward alignment should be a property of the generative model itself, not an afterthought, and redesign the model for adaptability. We propose "Diamond Maps", stochastic flow map models that enable efficient and accurate alignment to arbitrary rewards at inference time. Diamond Maps amortize many simulation steps into a single-step sampler, like flow maps, while preserving the stochasticity required for optimal reward alignment. This design makes search, sequential Monte Carlo, and guidance scalable by enabling efficient and consistent estimation of the value function. Our experiments show that Diamond Maps can be learned efficiently via distillation from GLASS Flows, achieve stronger reward alignment performance, and scale better than existing methods. Our results point toward a practical route to generative models that can be rapidly adapted to arbitrary preferences and constraints at inference time.

Related papers

• How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance [2.2602594453321063]
  Flow Map Reward Guidance (FMRG) is a training-free, single-trajectory framework that uses the flow map to both integrate and guide the flow.<n>At text-to-image scale, FMRG matches or surpasses baselines across inverse problems, style transfer, human preferences, and VLM rewards with as few as 3 NFEs.
  arXiv  Detail & Related papers  (2026-04-29T19:56:53Z)
• Euphonium: Steering Video Flow Matching via Process Reward Gradient Guided Stochastic Dynamics [49.242224984144904]
  We propose Euphonium, a novel framework that steers generation via process reward gradient guided dynamics.<n>Our key insight is to formulate the sampling process as a theoretically principled algorithm that explicitly incorporates the gradient of a Process Reward Model.<n>We derive a distillation objective that internalizes the guidance signal into the flow network, eliminating inference-time dependency on the reward model.
  arXiv  Detail & Related papers  (2026-02-04T08:59:57Z)
• Meta Flow Maps enable scalable reward alignment [22.109809498990362]
  Controlling generative models is computationally expensive.<n>This is because optimal alignment with a reward function--whether via inference-time steering or fine-tuning--requires estimating the value function.<n>We introduce Meta Flow Maps (MFMs), a framework consistency models and flow maps into extending the regime.
  arXiv  Detail & Related papers  (2026-01-20T19:39:56Z)
• Test-time scaling of diffusions with flow maps [68.79792714591564]
  A common recipe to improve diffusion models at test-time is to introduce the gradient of the reward into the dynamics of the diffusion itself.<n>We propose a simple solution by working directly with a flow map.<n>By exploiting a relationship between the flow map and velocity field governing the instantaneous transport, we construct an algorithm, Flow Map Trajectory Tilting (FMTT), which provably performs better ascent on the reward than standard test-time methods.
  arXiv  Detail & Related papers  (2025-11-27T18:44:12Z)
• DiffuMatch: Category-Agnostic Spectral Diffusion Priors for Robust Non-rigid Shape Matching [53.39693288324375]
  We show that both in-network regularization and functional map training can be replaced with data-driven methods.<n>We first train a generative model of functional maps in the spectral domain using score-based generative modeling.<n>We then exploit the resulting model to promote the structural properties of ground truth functional maps on new shape collections.
  arXiv  Detail & Related papers  (2025-07-31T16:44:54Z)
• Align Your Flow: Scaling Continuous-Time Flow Map Distillation [63.927438959502226]
  Flow maps connect any two noise levels in a single step and remain effective across all step counts.<n>We extensively validate our flow map models, called Align Your Flow, on challenging image generation benchmarks.<n>We show text-to-image flow map models that outperform all existing non-adversarially trained few-step samplers in text-conditioned synthesis.
  arXiv  Detail & Related papers  (2025-06-17T15:06:07Z)
• Asynchronously Trained Distributed Topographic Maps [0.0]
  We present an algorithm that uses $N$ autonomous units to generate a feature map by distributed training. Unit autonomy is achieved by sparse interaction in time & space through the combination of a distributed search, and a cascade-driven weight updating scheme.
  arXiv  Detail & Related papers  (2023-01-20T01:15:56Z)
• Smooth Non-Rigid Shape Matching via Effective Dirichlet Energy Optimization [46.30376601157526]
  We introduce pointwise map smoothness via the Dirichlet energy into the functional map pipeline. We propose an algorithm for optimizing it efficiently, which leads to high-quality results in challenging settings.
  arXiv  Detail & Related papers  (2022-10-05T14:07:17Z)
• Optimally Efficient Sequential Calibration of Binary Classifiers to Minimize Classification Error [0.0]
  We show that for the given target variables and the score outputs of an estimator, an 'optimal' soft mapping is a hard mapping that maps the score values to probabilities. We propose a sequential merger approach, which produces an 'optimal' hard mapping sequentially with each incoming new sample.
  arXiv  Detail & Related papers  (2021-08-19T16:42:40Z)
• HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps [81.86923212296863]
  HD maps are maps with precise definitions of road lanes with rich semantics of the traffic rules. There are only a small amount of real-world road topologies and geometries, which significantly limits our ability to test out the self-driving stack. We propose HDMapGen, a hierarchical graph generation model capable of producing high-quality and diverse HD maps.
  arXiv  Detail & Related papers  (2021-06-28T17:59:30Z)
• Learning Propagation Rules for Attribution Map Generation [146.71503336770886]
  We propose a dedicated method to generate attribution maps that allow us to learn the propagation rules automatically. Specifically, we introduce a learnable plugin module, which enables adaptive propagation rules for each pixel. The introduced learnable module can be trained under any auto-grad framework with higher-order differential support.
  arXiv  Detail & Related papers  (2020-10-14T16:23:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.