Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning

• URL: http://arxiv.org/abs/2306.04875v1
• Date: Thu, 8 Jun 2023 02:12:26 GMT
• Title: Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning
• Authors: Jifeng Hu, Yanchao Sun, Sili Huang, SiYuan Guo, Hechang Chen, Li Shen, Lichao Sun, Yi Chang, Dacheng Tao
• Abstract summary: We propose an effective temporally-conditional diffusion model coined Temporally-Composable diffuser (TCD) TCD extracts temporal information from interaction sequences and explicitly guides generation with temporal conditions. Our method reaches or matches the best performance compared with prior SOTA baselines.
• Score: 71.24316734338501
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent works have shown the potential of diffusion models in computer vision and natural language processing. Apart from the classical supervised learning fields, diffusion models have also shown strong competitiveness in reinforcement learning (RL) by formulating decision-making as sequential generation. However, incorporating temporal information of sequential data and utilizing it to guide diffusion models to perform better generation is still an open challenge. In this paper, we take one step forward to investigate controllable generation with temporal conditions that are refined from temporal information. We observe the importance of temporal conditions in sequential generation in sufficient explorative scenarios and provide a comprehensive discussion and comparison of different temporal conditions. Based on the observations, we propose an effective temporally-conditional diffusion model coined Temporally-Composable Diffuser (TCD), which extracts temporal information from interaction sequences and explicitly guides generation with temporal conditions. Specifically, we separate the sequences into three parts according to time expansion and identify historical, immediate, and prospective conditions accordingly. Each condition preserves non-overlapping temporal information of sequences, enabling more controllable generation when we jointly use them to guide the diffuser. Finally, we conduct extensive experiments and analysis to reveal the favorable applicability of TCD in offline RL tasks, where our method reaches or matches the best performance compared with prior SOTA baselines.

Related papers

• Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model [66.91323540178739]
  Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior. We revisit SR from a novel information-theoretic perspective and find that sequential modeling methods fail to adequately capture randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests.
  arXiv  Detail & Related papers  (2024-10-31T14:52:01Z)
• Retrieval-Augmented Diffusion Models for Time Series Forecasting [19.251274915003265]
  We propose a Retrieval- Augmented Time series Diffusion model (RATD) RATD consists of two parts: an embedding-based retrieval process and a reference-guided diffusion model. Our approach allows leveraging meaningful samples within the database to aid in sampling, thus maximizing the utilization of datasets.
  arXiv  Detail & Related papers  (2024-10-24T13:14:39Z)
• Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting [19.383395337330082]
  We propose a generic channel-aware Contrastive Conditional Diffusion model entitled CCDM. The proposed CCDM can exhibit superior forecasting capability compared to current state-of-the-art diffusion forecasters.
  arXiv  Detail & Related papers  (2024-10-03T03:13:15Z)
• Leveraging Priors via Diffusion Bridge for Time Series Generation [3.2066708654182743]
  Time series generation is widely used in real-world applications such as simulation, data augmentation, and hypothesis test techniques. diffusion models have emerged as the de facto approach for time series generation. TimeBridge is a framework that enables flexible synthesis by leveraging diffusion bridges to learn the transport between chosen prior and data distributions.
  arXiv  Detail & Related papers  (2024-08-13T06:47:59Z)
• A Survey on Diffusion Models for Time Series and Spatio-Temporal Data [92.1255811066468]
  We review the use of diffusion models in time series and S-temporal data, categorizing them by model, task type, data modality, and practical application domain. We categorize diffusion models into unconditioned and conditioned types discuss time series and S-temporal data separately. Our survey covers their application extensively in various fields including healthcare, recommendation, climate, energy, audio, and transportation.
  arXiv  Detail & Related papers  (2024-04-29T17:19:40Z)
• A Temporally Disentangled Contrastive Diffusion Model for Spatiotemporal Imputation [35.46631415365955]
  We introduce a conditional diffusion framework called C$2$TSD, which incorporates disentangled temporal (trend and seasonality) representations as conditional information. Our experiments on three real-world datasets demonstrate the superior performance of our approach compared to a number of state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-18T11:59:04Z)
• Add and Thin: Diffusion for Temporal Point Processes [24.4686728569167]
  ADD-THIN is a principled probabilistic denoising diffusion model for temporal point process (TPP) networks. It operates on entire event sequences and matches state-of-the-art TPP models in density estimation. In experiments on synthetic and real-world datasets, our model matches the state-of-the-art TPP models in density estimation and strongly outperforms them in forecasting.
  arXiv  Detail & Related papers  (2023-11-02T10:42:35Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
  We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
  arXiv  Detail & Related papers  (2022-12-20T17:40:03Z)
• Conditional Hybrid GAN for Sequence Generation [56.67961004064029]
  We propose a novel conditional hybrid GAN (C-Hybrid-GAN) to solve this issue. We exploit the Gumbel-Softmax technique to approximate the distribution of discrete-valued sequences. We demonstrate that the proposed C-Hybrid-GAN outperforms the existing methods in context-conditioned discrete-valued sequence generation.
  arXiv  Detail & Related papers  (2020-09-18T03:52:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.