ANT: Adaptive Noise Schedule for Time Series Diffusion Models

• URL: http://arxiv.org/abs/2410.14488v1
• Date: Fri, 18 Oct 2024 14:16:54 GMT
• Title: ANT: Adaptive Noise Schedule for Time Series Diffusion Models
• Authors: Seunghan Lee, Kibok Lee, Taeyoung Park,
• Abstract summary: We propose Adaptive Noise schedule for Time series diffusion models (ANT) It automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. We validate our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains.
• Score: 5.752266579415516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advances in diffusion models for generative artificial intelligence have recently propagated to the time series (TS) domain, demonstrating state-of-the-art performance on various tasks. However, prior works on TS diffusion models often borrow the framework of existing works proposed in other domains without considering the characteristics of TS data, leading to suboptimal performance. In this work, we propose Adaptive Noise schedule for Time series diffusion models (ANT), which automatically predetermines proper noise schedules for given TS datasets based on their statistics representing non-stationarity. Our intuition is that an optimal noise schedule should satisfy the following desiderata: 1) It linearly reduces the non-stationarity of TS data so that all diffusion steps are equally meaningful, 2) the data is corrupted to the random noise at the final step, and 3) the number of steps is sufficiently large. The proposed method is practical for use in that it eliminates the necessity of finding the optimal noise schedule with a small additional cost to compute the statistics for given datasets, which can be done offline before training. We validate the effectiveness of our method across various tasks, including TS forecasting, refinement, and generation, on datasets from diverse domains. Code is available at this repository: https://github.com/seunghan96/ANT.

Related papers

• Information-Guided Noise Allocation for Efficient Diffusion Training [50.49440376762109]
  InfoNoise is a principled data-adaptive training noise schedule that replaces schedule design with an information-guided noise sampling distribution.<n>Across natural-image benchmarks, InfoNoise matches or surpasses tuned EDM-style schedules, in some cases with a substantial training speedup.<n>Overall, InfoNoise makes noise scheduling data-adaptive, reducing the need for per-dataset schedule design as diffusion models expand across domains.
  arXiv  Detail & Related papers  (2026-02-20T22:45:01Z)
• ADiff4TPP: Asynchronous Diffusion Models for Temporal Point Processes [30.928368603673285]
  This work introduces a novel approach to modeling temporal point processes using diffusion models with an asynchronous noise schedule. We derive an objective to effectively train these models for a general family of noise schedules based on conditional flow matching. Our method achieves the joint distribution of the latent representations of events in a sequence and state-of-the-art results in predicting both the next inter-event time and event type on benchmark datasets.
  arXiv  Detail & Related papers  (2025-04-29T04:17:39Z)
• Beyond Fixed Horizons: A Theoretical Framework for Adaptive Denoising Diffusions [1.9116784879310031]
  We introduce a new class of generative diffusion models that achieve a time-homogeneous structure for both the noising and denoising processes.<n>A key feature of the model is its adaptability to the target data, enabling a variety of downstream tasks using a pre-trained unconditional generative model.
  arXiv  Detail & Related papers  (2025-01-31T18:23:27Z)
• Score-Optimal Diffusion Schedules [29.062842062257918]
  An appropriate discretisation schedule is crucial to obtain high quality samples. This paper presents a novel algorithm for adaptively selecting an optimal discretisation schedule. We find that our learned schedule recovers performant schedules previously only discovered through manual search.
  arXiv  Detail & Related papers  (2024-12-10T19:26:51Z)
• Constant Rate Schedule: Constant-Rate Distributional Change for Efficient Training and Sampling in Diffusion Models [16.863038973001483]
  We propose a noise schedule that ensures a constant rate of change in the probability distribution of diffused data throughout the diffusion process. The functional form of the noise schedule is automatically determined and tailored to each dataset and type of diffusion model.
  arXiv  Detail & Related papers  (2024-11-19T03:02:39Z)
• FM-TS: Flow Matching for Time Series Generation [71.31148785577085]
  We introduce FM-TS, a rectified Flow Matching-based framework for Time Series generation. FM-TS is more efficient in terms of training and inference. We have achieved superior performance in solar forecasting and MuJoCo imputation tasks.
  arXiv  Detail & Related papers  (2024-11-12T03:03:23Z)
• Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model [66.91323540178739]
  Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior. We revisit SR from a novel information-theoretic perspective and find that sequential modeling methods fail to adequately capture randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests.
  arXiv  Detail & Related papers  (2024-10-31T14:52:01Z)
• Retrieval-Augmented Diffusion Models for Time Series Forecasting [19.251274915003265]
  We propose a Retrieval- Augmented Time series Diffusion model (RATD) RATD consists of two parts: an embedding-based retrieval process and a reference-guided diffusion model. Our approach allows leveraging meaningful samples within the database to aid in sampling, thus maximizing the utilization of datasets.
  arXiv  Detail & Related papers  (2024-10-24T13:14:39Z)
• Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration [53.63593099509471]
  We propose a scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Exploration to train an additional scheduler model dedicated to scheduling contextualized noise for each sentence. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by our scheduler model for updating and generation.
  arXiv  Detail & Related papers  (2024-10-17T04:06:02Z)
• Improved Noise Schedule for Diffusion Training [51.849746576387375]
  We propose a novel approach to design the noise schedule for enhancing the training of diffusion models. We empirically demonstrate the superiority of our noise schedule over the standard cosine schedule.
  arXiv  Detail & Related papers  (2024-07-03T17:34:55Z)
• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Diffusion Models With Learned Adaptive Noise [12.530583016267768]
  In this paper, we explore whether the diffusion process can be learned from data. A widely held assumption is that the ELBO is invariant to the noise process. We propose MULAN, a learned diffusion process that applies noise at different rates across an image.
  arXiv  Detail & Related papers  (2023-12-20T18:00:16Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• The Optimal Noise in Noise-Contrastive Learning Is Not What You Think [80.07065346699005]
  We show that deviating from this assumption can actually lead to better statistical estimators. In particular, the optimal noise distribution is different from the data's and even from a different family.
  arXiv  Detail & Related papers  (2022-03-02T13:59:20Z)
• Voice2Series: Reprogramming Acoustic Models for Time Series Classification [65.94154001167608]
  Voice2Series is a novel end-to-end approach that reprograms acoustic models for time series classification. We show that V2S either outperforms or is tied with state-of-the-art methods on 20 tasks, and improves their average accuracy by 1.84%.
  arXiv  Detail & Related papers  (2021-06-17T07:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.