Time dependent loss reweighting for flow matching and diffusion models is theoretically justified

• URL: http://arxiv.org/abs/2511.16599v1
• Date: Thu, 20 Nov 2025 17:55:21 GMT
• Title: Time dependent loss reweighting for flow matching and diffusion models is theoretically justified
• Abstract summary: We show that the Bregman divergence loss and the linear parameterization of the generator can depend on both the current state $X_t$ and the time $t$.<n>We also show this for Edit Flows, which falls outside of Generator Matching.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This brief note clarifies that, in Generator Matching (which subsumes a large family of flow matching and diffusion models over continuous, manifold, and discrete spaces), both the Bregman divergence loss and the linear parameterization of the generator can depend on both the current state $X_t$ and the time $t$, and we show that the expectation over time in the loss can be taken with respect to a broad class of time distributions. We also show this for Edit Flows, which falls outside of Generator Matching. That the loss can depend on $t$ clarifies that time-dependent loss weighting schemes, often used in practice to stabilize training, are theoretically justified when the specific flow or diffusion scheme is a special case of Generator Matching (or Edit Flows). It also often simplifies the construction of $X_1$-predictor schemes, which are sometimes preferred for model-related reasons. We show examples that rely upon the dependence of linear parameterizations, and of the Bregman divergence loss, on $t$ and $X_t$.

Related papers

• Generative Diffusion Surrogates with Analytical Variance Schedule [0.0]
  Transport describes physical systems in which an initially structured distribution spreads under unresolved forcing, scattering, or heterogeneous media.<n>Here we prescribe the forward noising rate as the time derivative of this variance, turning generative time into a calibrated transport clock.<n>For ballistic-to-diffusive transport in turbulent plasmas, the surrogate matches test-particle distributions, reproduces the laboratory-measured variance scale, and tracks the simulated kurtosis evolution without schedule tuning.
  arXiv  Detail & Related papers  (2026-09-01T18:00:00Z)
• Heavy-Tailed Flow Matching via Random Clocks [52.78194016835051]
  Heavy-tailed data arise in many domains where rare events carry disproportionate importance, such as imbalanced image datasets, financial returns, and weather extremes.<n>We propose Heavy-Tailed Flow Matching via Random Clocks (HTFM), a framework that portrays heavy-tailed sources as mixtures of clock-conditioned Gaussian sources.
  arXiv  Detail & Related papers  (2026-07-15T13:48:05Z)
• Joint distribution of upstream runoff governs downstream river-discharge prediction uncertainty in distributed ML models [0.0]
  This technical note argues that moving probabilistic prediction from lumped to distributed models introduces a specific new requirement.<n>In lumped inference, the model predicts the outlet distribution directly and can modulate spread from basin attributes.<n>In distributed inference, downstream discharge is obtained by routing many upstream runoff predictions, so independent local sampling averages uncertainty away.
  arXiv  Detail & Related papers  (2026-07-03T11:31:57Z)
• Probability-Conserving Flow Guidance [49.03107678763765]
  Adaptive Manifold Guidance (AdaMaG) improves realism, reduces hallucinations, and controlled desaturation in high-guidance.<n>Across image generation benchmarks, AdaMaG improves realism, reduces hallucinations, and controlled desaturation in high-guidance.
  arXiv  Detail & Related papers  (2026-05-19T16:34:01Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• Dynamical Regimes of Multimodal Diffusion Models [0.0]
  We present a theoretical framework for coupled diffusion models, using coupled Ornstein-Uhlenbeck processes as a tractable model.<n>A key prediction is the synchronization gap'', a temporal window during the reverse generative process where distinct eigenmodes stabilize at different rates.<n>We show that the coupling strength acts as a spectral filter that enforces a tunable temporal hierarchy on generation.
  arXiv  Detail & Related papers  (2026-02-04T17:16:12Z)
• An Elementary Approach to Scheduling in Generative Diffusion Models [55.171367482496755]
  An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed.<n> Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies.
  arXiv  Detail & Related papers  (2026-01-20T05:06:26Z)
• Dimension-free error estimate for diffusion model and optimal scheduling [22.20348860913421]
  Diffusion generative models have emerged as powerful tools for producing synthetic data from an empirically observed distribution.<n>Previous analyses have quantified the error between the generated and the true data distributions in terms of Wasserstein distance or Kullback-Leibler divergence.<n>In this work, we derive an explicit, dimension-free bound on the discrepancy between the generated and the true data distributions.
  arXiv  Detail & Related papers  (2025-12-01T15:58:20Z)
• Convergence, design and training of continuous-time dropout as a random batch method [0.0]
  We study dropout regularization in continuous-time models through the lens of random-batch methods.<n>We construct an unbiased, well-posed estimator that mimics dropout by sampling neuron batches over time intervals of length $h$.<n>We then specialize in a single-layer neural ODE and validate the theory on classification and flow matching.
  arXiv  Detail & Related papers  (2025-10-15T04:19:01Z)
• Error Analysis of Discrete Flow with Generator Matching [34.847768962128164]
  We develop a unified framework grounded in calculus theory to investigate the theoretical properties of discrete flow models.<n>Specifically, we derive the KL divergence of two path measures regarding two continuous-time Markov chains with different transition rates.<n>We provide a comprehensive analysis that encompasses the error arising from transition rate estimation and early stopping.
  arXiv  Detail & Related papers  (2025-09-26T05:41:45Z)
• Flow based approach for Dynamic Temporal Causal models with non-Gaussian or Heteroscedastic Noises [37.02662517645979]
  We introduce FANTOM, a unified framework for causal discovery.<n>It handles non-stationary processes along with non-Gaussian and heteroscedastic noises.<n>It simultaneously infers the number of regimes and their corresponding indices and learns each regime's Directed Acyclic Graph.
  arXiv  Detail & Related papers  (2025-06-20T15:12:43Z)
• Trajectory Generator Matching for Time Series [0.0]
  We find new generators of SDEs and jump processes inspired by trajectory flow matching.<n>We can handle discontinuities of the underlying processes by parameterizing the jump kernel densities.<n>Unlike most other approaches, we are able to handle irregularly sampled time series.
  arXiv  Detail & Related papers  (2025-05-29T07:56:32Z)
• EquiReg: Equivariance Regularized Diffusion for Inverse Problems [67.01847869495558]
  We propose EquiReg diffusion, a framework for regularizing posterior sampling in diffusion-based inverse problem solvers.<n>When applied to a variety of solvers, EquiReg outperforms state-of-the-art diffusion models in both linear and nonlinear image restoration tasks.
  arXiv  Detail & Related papers  (2025-05-29T01:25:43Z)
• Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat [49.1574468325115]
  This paper presents explicit non-asymptotic bounds on the forward diffusion error in total variation (TV)<n>We parametrise multi-modal data distributions in terms of the distance $R$ to their furthest modes and consider forward diffusions with additive and multiplicative noise.
  arXiv  Detail & Related papers  (2024-08-25T10:28:31Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• On Error Propagation of Diffusion Models [77.91480554418048]
  We develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs. We apply the cumulative error as a regularization term to reduce error propagation. Our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.
  arXiv  Detail & Related papers  (2023-08-09T15:31:17Z)
• Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecasting [14.951166842027819]
  We propose a dynamic mixture of zero-mean Gaussian distributions for the time-varying error process.<n>The proposed method can be seamlessly integrated into existing deep-learning frameworks with only a few additional parameters to be learned.<n>We evaluate the proposed method on a traffic speed forecasting task and find that our method not only improves model horizons but also provides interpretabletemporal correlation structures.
  arXiv  Detail & Related papers  (2022-12-10T22:50:00Z)
• Multivariate Probabilistic Forecasting of Intraday Electricity Prices using Normalizing Flows [62.997667081978825]
  In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the EPEX spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts.
  arXiv  Detail & Related papers  (2022-05-27T08:38:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.