Entropic Time Schedulers for Generative Diffusion Models

• URL: http://arxiv.org/abs/2504.13612v1
• Date: Fri, 18 Apr 2025 10:35:19 GMT
• Title: Entropic Time Schedulers for Generative Diffusion Models
• Authors: Dejan Stancevic, Luca Ambrogioni,
• Abstract summary: We present a time scheduler that selects sampling points based on entropy rather than uniform time spacing.<n>We show that using the (rescaled) entropic times greatly improves the inference performance of trained models.
• Score: 5.44478242486351
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The practical performance of generative diffusion models depends on the appropriate choice of the noise scheduling function, which can also be equivalently expressed as a time reparameterization. In this paper, we present a time scheduler that selects sampling points based on entropy rather than uniform time spacing, ensuring that each point contributes an equal amount of information to the final generation. We prove that this time reparameterization does not depend on the initial choice of time. Furthermore, we provide a tractable exact formula to estimate this \emph{entropic time} for a trained model using the training loss without substantial overhead. Alongside the entropic time, inspired by the optimality results, we introduce a rescaled entropic time. In our experiments with mixtures of Gaussian distributions and ImageNet, we show that using the (rescaled) entropic times greatly improves the inference performance of trained models. In particular, we found that the image quality in pretrained EDM2 models, as evaluated by FID and FD-DINO scores, can be substantially increased by the rescaled entropic time reparameterization without increasing the number of function evaluations, with greater improvements in the few NFEs regime.

Related papers

• Low-Dimensional Adaptation of Rectified Flow: A New Perspective through the Lens of Diffusion and Stochastic Localization [59.04314685837778]
  Rectified flow (RF) has gained considerable popularity due to its generation efficiency and state-of-the-art performance.<n>In this paper, we investigate the degree to which RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling.<n>We show that, using a carefully designed choice of the time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an complexity of order $O(k/varepsilon)$.
  arXiv  Detail & Related papers  (2026-01-21T22:09:27Z)
• TimeFlow: Towards Stochastic-Aware and Efficient Time Series Generation via Flow Matching Modeling [2.74279932215302]
  Time series data has emerged as a critical research topic due to its broad utility in supporting downstream time series mining tasks.<n>We propose TimeFlow, a novel flow matching framework that integrates a encoder-only architecture.<n>Our model consistently outperforms strong baselines in generation quality, diversity, and efficiency.
  arXiv  Detail & Related papers  (2025-11-11T08:28:26Z)
• Generalized Gaussian Entropy Model for Point Cloud Attribute Compression with Dynamic Likelihood Intervals [4.230780744307392]
  entropy parameters estimated by neural networks in current methods can be used for more accurate probability estimation.<n>We propose Mean Error Discriminator (MED) to determine whether the entropy parameter estimation is accurate and then dynamically adjust likelihood intervals.<n>Experiments show that our method significantly improves rate-distortion (RD) performance on three VAE-based models for point cloud attribute compression.
  arXiv  Detail & Related papers  (2025-06-11T08:34:56Z)
• MATEY: multiscale adaptive foundation models for spatiotemporal physical systems [2.7767126393602726]
  We propose two adaptive tokenization schemes that dynamically adjust patch sizes based on local features.<n>We evaluate the performance of a proposed multiscale adaptive model, MATEY, in a sequence of experiments.<n>We also demonstrate fine-tuning tasks featuring different physics that models pretrained on PDE data.
  arXiv  Detail & Related papers  (2024-12-29T22:13:16Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics.<n>We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization [97.35427957922714]
  We present an algorithm named pairwise sample optimization (PSO), which enables the direct fine-tuning of an arbitrary timestep-distilled diffusion model.<n>PSO introduces additional reference images sampled from the current time-step distilled model, and increases the relative likelihood margin between the training images and reference images.<n>We show that PSO can directly adapt distilled models to human-preferred generation with both offline and online-generated pairwise preference image data.
  arXiv  Detail & Related papers  (2024-10-04T07:05:16Z)
• Score-based Generative Models with Adaptive Momentum [40.84399531998246]
  We propose an adaptive momentum sampling method to accelerate the transforming process. We show that our method can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up.
  arXiv  Detail & Related papers  (2024-05-22T15:20:27Z)
• Real-time Inference and Extrapolation via a Diffusion-inspired Temporal Transformer Operator (DiTTO) [1.5728609542259502]
  We propose an operator learning method to solve time-dependent partial differential equations (PDEs) continuously and with extrapolation in time without any temporal discretization. The proposed method, named Diffusion-inspired Temporal Transformer Operator (DiTTO), is inspired by latent diffusion models and their conditioning mechanism. We demonstrate its extrapolation capability on a climate problem by estimating the temperature around the globe for several years, and also in modeling hypersonic flows around a double-cone.
  arXiv  Detail & Related papers  (2023-07-18T08:45:54Z)
• Gated Recurrent Neural Networks with Weighted Time-Delay Feedback [59.125047512495456]
  We introduce a novel gated recurrent unit (GRU) with a weighted time-delay feedback mechanism. We show that $tau$-GRU can converge faster and generalize better than state-of-the-art recurrent units and gated recurrent architectures.
  arXiv  Detail & Related papers  (2022-12-01T02:26:34Z)
• FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels [82.53569355337586]
  This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support. The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG) Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-10T12:35:02Z)
• Leveraging Image-based Generative Adversarial Networks for Time Series Generation [4.541582055558865]
  We propose a two-dimensional image representation for time series, the Extended Intertemporal Return Plot (XIRP) Our approach captures the intertemporal time series dynamics in a scale-invariant and invertible way, reducing training time and improving sample quality.
  arXiv  Detail & Related papers  (2021-12-15T11:55:11Z)
• Moment evolution equations and moment matching for stochastic image EPDiff [68.97335984455059]
  Models of image deformation allow study of time-continuous effects transforming images by deforming the image domain. Applications include medical image analysis with both population trends and random subject specific variation. We use moment approximations of the corresponding Ito diffusion to construct estimators for statistical inference in the parameters full model.
  arXiv  Detail & Related papers  (2021-10-07T11:08:11Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• The temporal overfitting problem with applications in wind power curve modeling [8.057262184815636]
  We propose a new method to tackle the temporal overfitting problem. Our specific application in this paper targets the power curve modeling in wind energy.
  arXiv  Detail & Related papers  (2020-12-02T17:39:57Z)
• Score-Based Generative Modeling through Stochastic Differential Equations [114.39209003111723]
  We present a differential equation that transforms a complex data distribution to a known prior distribution by injecting noise. A corresponding reverse-time SDE transforms the prior distribution back into the data distribution by slowly removing the noise. By leveraging advances in score-based generative modeling, we can accurately estimate these scores with neural networks. We demonstrate high fidelity generation of 1024 x 1024 images for the first time from a score-based generative model.
  arXiv  Detail & Related papers  (2020-11-26T19:39:10Z)
• Efficient Variational Bayesian Structure Learning of Dynamic Graphical Models [19.591265962713837]
  Estimating time-varying graphical models is of paramount importance in various social, financial, biological, and engineering systems. Existing methods require extensive tuning of parameters that control the graph sparsity and temporal smoothness. We propose a low-complexity tuning-free Bayesian approach, named BADGE.
  arXiv  Detail & Related papers  (2020-09-16T14:19:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.