Abstract: In many cases, it is difficult to generate highly accurate models for time
series data using a known parametric model structure. In response, an
increasing body of research focuses on using neural networks to model time
series approximately. A common assumption in training neural networks on time
series is that the errors at different time steps are uncorrelated. However,
due to the temporality of the data, errors are actually autocorrelated in many
cases, which makes such maximum likelihood estimation inaccurate. In this
paper, we propose to learn the autocorrelation coefficient jointly with the
model parameters in order to adjust for autocorrelated errors. For time series
regression, large-scale experiments indicate that our method outperforms the
Prais-Winsten method, especially when the autocorrelation is strong.
Furthermore, we broaden our method to time series forecasting and apply it with
various state-of-the-art models. Results across a wide range of real-world
datasets show that our method enhances performance in almost all cases.