Related papers: Partial-Multivariate Model for Forecasting

Partial-Multivariate Model for Forecasting

URL: http://arxiv.org/abs/2408.09703v1
Date: Mon, 19 Aug 2024 05:18:50 GMT
Title: Partial-Multivariate Model for Forecasting
Authors: Jaehoon Lee, Hankook Lee, Sungik Choi, Sungjun Cho, Moontae Lee,
Abstract summary: We propose a Transformer-based partial-multivariate model, PMformer, for forecasting problems. We demonstrate that PMformer outperforms various univariate and complete-multivariate models. We also highlight other advantages of PMformer: efficiency and robustness under missing features.
Score: 28.120094495344453
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: When solving forecasting problems including multiple time-series features, existing approaches often fall into two extreme categories, depending on whether to utilize inter-feature information: univariate and complete-multivariate models. Unlike univariate cases which ignore the information, complete-multivariate models compute relationships among a complete set of features. However, despite the potential advantage of leveraging the additional information, complete-multivariate models sometimes underperform univariate ones. Therefore, our research aims to explore a middle ground between these two by introducing what we term Partial-Multivariate models where a neural network captures only partial relationships, that is, dependencies within subsets of all features. To this end, we propose PMformer, a Transformer-based partial-multivariate model, with its training algorithm. We demonstrate that PMformer outperforms various univariate and complete-multivariate models, providing a theoretical rationale and empirical analysis for its superiority. Additionally, by proposing an inference technique for PMformer, the forecasting accuracy is further enhanced. Finally, we highlight other advantages of PMformer: efficiency and robustness under missing features.

Related papers

Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
arXiv Detail & Related papers (2024-02-05T03:25:04Z)
Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives [5.549794481031468]
Devising deep latent variable models for multi-modal data has been a long-standing theme in machine learning research. In this work, we consider a variational objective that can tightly approximate the data log-likelihood. We develop more flexible aggregation schemes that avoid the inductive biases in PoE or MoE approaches.
arXiv Detail & Related papers (2023-09-01T10:32:21Z)
Generative machine learning methods for multivariate ensemble post-processing [2.266704492832475]
We present a novel class of nonparametric data-driven distributional regression models based on generative machine learning. In two case studies, our generative model shows significant improvements over state-of-the-art methods.
arXiv Detail & Related papers (2022-09-26T09:02:30Z)
CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting [70.54920804222031]
We propose a general probabilistic multi-view forecasting framework CAMul. It can learn representations and uncertainty from diverse data sources. It integrates the knowledge and uncertainty from each data view in a dynamic context-specific manner. We show that CAMul outperforms other state-of-art probabilistic forecasting models by over 25% in accuracy and calibration.
arXiv Detail & Related papers (2021-09-15T17:13:47Z)
Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
arXiv Detail & Related papers (2021-07-07T11:17:09Z)
Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
arXiv Detail & Related papers (2020-10-25T18:51:15Z)
Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
arXiv Detail & Related papers (2020-07-21T08:18:06Z)
Bayesian Sparse Factor Analysis with Kernelized Observations [67.60224656603823]
Multi-view problems can be faced with latent variable models. High-dimensionality and non-linear issues are traditionally handled by kernel methods. We propose merging both approaches into single model.
arXiv Detail & Related papers (2020-06-01T14:25:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.