Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting

• URL: http://arxiv.org/abs/2211.15092v2
• Date: Thu, 2 Nov 2023 07:25:18 GMT
• Title: Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting
• Authors: Arindam Jati, Vijay Ekambaram, Shaonli Pal, Brian Quanz, Wesley M. Gifford, Pavithra Harsha, Stuart Siegel, Sumanta Mukherjee, Chandra Narayanaswami
• Abstract summary: We propose a novel technique, H-Pro, to drive HPO via test proxies by exploiting data hierarchies associated with time series datasets. H-Pro can be applied on any off-the-shelf machine learning model to perform HPO. Our approach outperforms existing state-of-the-art methods in Tourism, Wiki, and Traffic datasets.
• Score: 9.906423777470737
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Selecting the right set of hyperparameters is crucial in time series forecasting. The classical temporal cross-validation framework for hyperparameter optimization (HPO) often leads to poor test performance because of a possible mismatch between validation and test periods. To address this test-validation mismatch, we propose a novel technique, H-Pro to drive HPO via test proxies by exploiting data hierarchies often associated with time series datasets. Since higher-level aggregated time series often show less irregularity and better predictability as compared to the lowest-level time series which can be sparse and intermittent, we optimize the hyperparameters of the lowest-level base-forecaster by leveraging the proxy forecasts for the test period generated from the forecasters at higher levels. H-Pro can be applied on any off-the-shelf machine learning model to perform HPO. We validate the efficacy of our technique with extensive empirical evaluation on five publicly available hierarchical forecasting datasets. Our approach outperforms existing state-of-the-art methods in Tourism, Wiki, and Traffic datasets, and achieves competitive result in Tourism-L dataset, without any model-specific enhancements. Moreover, our method outperforms the winning method of the M5 forecast accuracy competition.

Related papers

• Optimal starting point for time series forecasting [1.9937737230710553]
  We introduce a novel approach called Optimal Starting Point Time Series Forecast (OSP-TSP) By adjusting the sequence length via leveraging the XGBoost and LightGBM models, the proposed approach can determine optimal starting point (OSP) of the time series. Empirical results indicate that predictions based on the OSP-TSP approach consistently outperform those using the complete dataset.
  arXiv  Detail & Related papers  (2024-09-25T11:51:00Z)
• MCDFN: Supply Chain Demand Forecasting via an Explainable Multi-Channel Data Fusion Network Model [0.0]
  We introduce the Multi-Channel Data Fusion Network (MCDFN), a hybrid architecture that integrates CNN, Long Short-Term Memory networks (LSTM), and Gated Recurrent Units (GRU) Our comparative benchmarking demonstrates that MCDFN outperforms seven other deep-learning models. This research advances demand forecasting methodologies and offers practical guidelines for integrating MCDFN into supply chain systems.
  arXiv  Detail & Related papers  (2024-05-24T14:30:00Z)
• Loss Shaping Constraints for Long-Term Time Series Forecasting [79.3533114027664]
  We present a Constrained Learning approach for long-term time series forecasting that respects a user-defined upper bound on the loss at each time-step. We propose a practical Primal-Dual algorithm to tackle it, and aims to demonstrate that it exhibits competitive average performance in time series benchmarks, while shaping the errors across the predicted window.
  arXiv  Detail & Related papers  (2024-02-14T18:20:44Z)
• PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction [27.434939269672288]
  We use PAC-Bayes theory to obtain generalization bounds on the coverage and the efficiency of set-valued predictors. We leverage these theoretical results to provide a practical algorithm for using calibration data to fine-tune the parameters of a model and score function. We evaluate the approach on regression and classification tasks, and outperform baselines calibrated using a Hoeffding bound-based PAC guarantee on ICP.
  arXiv  Detail & Related papers  (2023-12-07T19:40:44Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
  High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach. Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
  arXiv  Detail & Related papers  (2023-09-12T13:28:06Z)
• Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning [73.75282761503581]
  We propose DiffTPT, which leverages pre-trained diffusion models to generate diverse and informative new data. Our experiments on test datasets with distribution shifts and unseen categories demonstrate that DiffTPT improves the zero-shot accuracy by an average of 5.13%.
  arXiv  Detail & Related papers  (2023-08-11T09:36:31Z)
• HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts [26.205660967039087]
  We use the lexicographic priority order on average validation loss and worst-case validation loss over chronological validation sets. We show the strong empirical performance of the proposed method on multiple machine learning tasks with temporal distribution shifts.
  arXiv  Detail & Related papers  (2023-05-28T19:41:23Z)
• Forecast reconciliation for vaccine supply chain optimization [61.13962963550403]
  Vaccine supply chain optimization can benefit from hierarchical time series forecasting. Forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts. We tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series.
  arXiv  Detail & Related papers  (2023-05-02T14:34:34Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Optimal Latent Space Forecasting for Large Collections of Short Time Series Using Temporal Matrix Factorization [0.0]
  It is a common practice to evaluate multiple methods and choose one of these methods or an ensemble for producing the best forecasts. We propose a framework for forecasting short high-dimensional time series data by combining low-rank temporal matrix factorization and optimal model selection on latent time series.
  arXiv  Detail & Related papers  (2021-12-15T11:39:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.