Temporal Domain Generalization with Drift-Aware Dynamic Neural Network

• URL: http://arxiv.org/abs/2205.10664v1
• Date: Sat, 21 May 2022 20:01:31 GMT
• Title: Temporal Domain Generalization with Drift-Aware Dynamic Neural Network
• Authors: Guangji Bai, Ling Chen, Liang Zhao
• Abstract summary: We propose a Temporal Domain Generalization with Drift-Aware Dynamic Neural Network (DRAIN) framework. Specifically, we formulate the problem into a Bayesian framework that jointly models the relation between data and model dynamics. It captures the temporal drift of model parameters and data distributions and can predict models in the future without the presence of future data.
• Score: 12.483886657900525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal domain generalization is a promising yet extremely challenging area where the goal is to learn models under temporally changing data distributions and generalize to unseen data distributions following the trends of the change. The advancement of this area is challenged by: 1) characterizing data distribution drift and its impacts on models, 2) expressiveness in tracking the model dynamics, and 3) theoretical guarantee on the performance. To address them, we propose a Temporal Domain Generalization with Drift-Aware Dynamic Neural Network (DRAIN) framework. Specifically, we formulate the problem into a Bayesian framework that jointly models the relation between data and model dynamics. We then build a recurrent graph generation scenario to characterize the dynamic graph-structured neural networks learned across different time points. It captures the temporal drift of model parameters and data distributions and can predict models in the future without the presence of future data. In addition, we explore theoretical guarantees of the model performance under the challenging temporal DG setting and provide theoretical analysis, including uncertainty and generalization error. Finally, extensive experiments on several real-world benchmarks with temporal drift demonstrate the effectiveness and efficiency of the proposed method.

Related papers

• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• Modeling Randomly Observed Spatiotemporal Dynamical Systems [7.381752536547389]
  Currently available neural network-based modeling approaches fall short when faced with data collected randomly over time and space. In response, we developed a new method that effectively handles such randomly sampled data. Our model integrates techniques from amortized variational inference, neural differential equations, neural point processes, and implicit neural representations to predict both the dynamics of the system and the timings and locations of future observations.
  arXiv  Detail & Related papers  (2024-06-01T09:03:32Z)
• A Survey on Diffusion Models for Time Series and Spatio-Temporal Data [92.1255811066468]
  We review the use of diffusion models in time series and S-temporal data, categorizing them by model, task type, data modality, and practical application domain. We categorize diffusion models into unconditioned and conditioned types discuss time series and S-temporal data separately. Our survey covers their application extensively in various fields including healthcare, recommendation, climate, energy, audio, and transportation.
  arXiv  Detail & Related papers  (2024-04-29T17:19:40Z)
• Out-of-Distribution Generalized Dynamic Graph Neural Network with Disentangled Intervention and Invariance Promotion [61.751257172868186]
  Dynamic graph neural networks (DyGNNs) have demonstrated powerful predictive abilities by exploiting graph and temporal dynamics. Existing DyGNNs fail to handle distribution shifts, which naturally exist in dynamic graphs.
  arXiv  Detail & Related papers  (2023-11-24T02:42:42Z)
• Exploring Time Granularity on Temporal Graphs for Dynamic Link Prediction in Real-world Networks [0.48346848229502226]
  Dynamic Graph Neural Networks (DGNNs) have emerged as the predominant approach for processing dynamic graph-structured data. In this paper, we explore the impact of time granularity when training DGNNs on dynamic graphs through extensive experiments.
  arXiv  Detail & Related papers  (2023-11-21T00:34:53Z)
• CODA: Temporal Domain Generalization via Concept Drift Simulator [34.21255368783787]
  In real-world applications, machine learning models often become obsolete due to shifts in the joint distribution arising from underlying temporal trends. We propose the COncept Drift simulAtor framework incorporating a predicted feature correlation matrix to simulate future data for model training.
  arXiv  Detail & Related papers  (2023-10-02T18:04:34Z)
• MemDA: Forecasting Urban Time Series with Memory-based Drift Adaptation [24.284969264008733]
  We propose a new urban time series prediction model for the concept drift problem, which encodes the drift by considering the periodicity in the data. Our design significantly outperforms state-of-the-art methods and can be well generalized to existing prediction backbones.
  arXiv  Detail & Related papers  (2023-09-25T15:22:28Z)
• Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatio-temporal Forecasting [60.03169701753824]
  We propose a novel Dynamic Diffusion-al Graph Neural Network (DVGNN) fortemporal forecasting. The proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding Root Mean Squared Error result.
  arXiv  Detail & Related papers  (2023-05-16T11:38:19Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Disentangled Generative Models for Robust Prediction of System Dynamics [2.6424064030995957]
  In this work, we treat the domain parameters of dynamical systems as factors of variation of the data generating process. By leveraging ideas from supervised disentanglement and causal factorization, we aim to separate the domain parameters from the dynamics in the latent space of generative models. Results indicate that disentangled VAEs adapt better to domain parameters spaces that were not present in the training data.
  arXiv  Detail & Related papers  (2021-08-26T09:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.