SutraNets: Sub-series Autoregressive Networks for Long-Sequence,
Probabilistic Forecasting
- URL: http://arxiv.org/abs/2312.14880v1
- Date: Fri, 22 Dec 2023 18:00:17 GMT
- Title: SutraNets: Sub-series Autoregressive Networks for Long-Sequence,
Probabilistic Forecasting
- Authors: Shane Bergsma, Timothy Zeyl, Lei Guo
- Abstract summary: SutraNets is a novel method for neural probabilistic forecasting of long-sequence time series.
It uses an autoregressive generative model to factorize the likelihood of long sequences into products of conditional probabilities.
We find SutraNets to significantly improve forecasting accuracy over competitive alternatives on six real-world datasets.
- Score: 4.815881393263451
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose SutraNets, a novel method for neural probabilistic forecasting of
long-sequence time series. SutraNets use an autoregressive generative model to
factorize the likelihood of long sequences into products of conditional
probabilities. When generating long sequences, most autoregressive approaches
suffer from harmful error accumulation, as well as challenges in modeling
long-distance dependencies. SutraNets treat long, univariate prediction as
multivariate prediction over lower-frequency sub-series. Autoregression
proceeds across time and across sub-series in order to ensure coherent
multivariate (and, hence, high-frequency univariate) outputs. Since sub-series
can be generated using fewer steps, SutraNets effectively reduce error
accumulation and signal path distances. We find SutraNets to significantly
improve forecasting accuracy over competitive alternatives on six real-world
datasets, including when we vary the number of sub-series and scale up the
depth and width of the underlying sequence models.
Related papers
- Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation
for Time Series [49.992908221544624]
Time series data often exhibit numerous missing values, which is the time series imputation task.
Previous deep learning methods have been shown to be effective for time series imputation.
We propose a non-generative time series imputation method that produces accurate imputations with inherent uncertainty.
arXiv Detail & Related papers (2023-12-03T05:52:30Z) - MPPN: Multi-Resolution Periodic Pattern Network For Long-Term Time
Series Forecasting [19.573651104129443]
Long-term time series forecasting plays an important role in various real-world scenarios.
Recent deep learning methods for long-term series forecasting tend to capture the intricate patterns of time series by decomposition-based or sampling-based methods.
We propose a novel deep learning network architecture, named Multi-resolution Periodic Pattern Network (MPPN), for long-term series forecasting.
arXiv Detail & Related papers (2023-06-12T07:00:37Z) - SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking [60.109453252858806]
A maximum-likelihood (MLE) objective does not match a downstream use-case of autoregressively generating high-quality sequences.
We formulate sequence generation as an imitation learning (IL) problem.
This allows us to minimize a variety of divergences between the distribution of sequences generated by an autoregressive model and sequences from a dataset.
Our resulting method, SequenceMatch, can be implemented without adversarial training or architectural changes.
arXiv Detail & Related papers (2023-06-08T17:59:58Z) - Generative Time Series Forecasting with Diffusion, Denoise, and
Disentanglement [51.55157852647306]
Time series forecasting has been a widely explored task of great importance in many applications.
It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series.
We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
arXiv Detail & Related papers (2023-01-08T12:20:46Z) - Layer Ensembles [95.42181254494287]
We introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network.
We show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run.
arXiv Detail & Related papers (2022-10-10T17:52:47Z) - STING: Self-attention based Time-series Imputation Networks using GAN [4.052758394413726]
STING (Self-attention based Time-series Imputation Networks using GAN) is proposed.
We take advantage of generative adversarial networks and bidirectional recurrent neural networks to learn latent representations of the time series.
Experimental results on three real-world datasets demonstrate that STING outperforms the existing state-of-the-art methods in terms of imputation accuracy.
arXiv Detail & Related papers (2022-09-22T06:06:56Z) - Sequence Length is a Domain: Length-based Overfitting in Transformer
Models [0.0]
In machine translation, the neural-based systems perform worse on very long sequences when compared to the preceding phrase-based translation approaches.
We show that the observed drop in performance is due to the hypothesis length corresponding to the lengths seen by the model during training rather than the length of the input sequence.
arXiv Detail & Related papers (2021-09-15T13:25:19Z) - Synergetic Learning of Heterogeneous Temporal Sequences for
Multi-Horizon Probabilistic Forecasting [48.8617204809538]
We propose Variational Synergetic Multi-Horizon Network (VSMHN), a novel deep conditional generative model.
To learn complex correlations across heterogeneous sequences, a tailored encoder is devised to combine the advances in deep point processes models and variational recurrent neural networks.
Our model can be trained effectively using variational inference and generates predictions with Monte-Carlo simulation.
arXiv Detail & Related papers (2021-01-31T11:00:55Z) - Short-Term Memory Optimization in Recurrent Neural Networks by
Autoencoder-based Initialization [79.42778415729475]
We explore an alternative solution based on explicit memorization using linear autoencoders for sequences.
We show how such pretraining can better support solving hard classification tasks with long sequences.
We show that the proposed approach achieves a much lower reconstruction error for long sequences and a better gradient propagation during the finetuning phase.
arXiv Detail & Related papers (2020-11-05T14:57:16Z) - Improved Predictive Deep Temporal Neural Networks with Trend Filtering [22.352437268596674]
We propose a new prediction framework based on deep neural networks and a trend filtering.
We reveal that the predictive performance of deep temporal neural networks improves when the training data is temporally processed by a trend filtering.
arXiv Detail & Related papers (2020-10-16T08:29:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.