Conformal PID Control for Time Series Prediction

• URL: http://arxiv.org/abs/2307.16895v1
• Date: Mon, 31 Jul 2023 17:59:16 GMT
• Title: Conformal PID Control for Time Series Prediction
• Authors: Anastasios N. Angelopoulos, Emmanuel J. Candes, Ryan J. Tibshirani
• Abstract summary: We study the problem of uncertainty quantification for time series prediction. We present algorithms that prospectively model conformal scores in an online setting. We also run experiments on predicting electricity demand, market returns, and temperature.
• Score: 10.992151305603265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the problem of uncertainty quantification for time series prediction, with the goal of providing easy-to-use algorithms with formal guarantees. The algorithms we present build upon ideas from conformal prediction and control theory, are able to prospectively model conformal scores in an online setting, and adapt to the presence of systematic errors due to seasonality, trends, and general distribution shifts. Our theory both simplifies and strengthens existing analyses in online conformal prediction. Experiments on 4-week-ahead forecasting of statewide COVID-19 death counts in the U.S. show an improvement in coverage over the ensemble forecaster used in official CDC communications. We also run experiments on predicting electricity demand, market returns, and temperature using autoregressive, Theta, Prophet, and Transformer models. We provide an extendable codebase for testing our methods and for the integration of new algorithms, data sets, and forecasting rules.

Related papers

• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs. We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• Conformal online model aggregation [29.43493007296859]
  This paper proposes a new approach towards conformal model aggregation in online settings. It is based on combining the prediction sets from several algorithms by voting, where weights on the models are adapted over time based on past performance.
  arXiv  Detail & Related papers  (2024-03-22T15:40:06Z)
• Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shift [28.73747033245012]
  We introduce a universal calibration methodology for the detection and adaptation of context-driven distribution shifts. A novel CDS detector, termed the "residual-based CDS detector" or "Reconditionor", quantifies the model's vulnerability to CDS. A high Reconditionor score indicates a severe susceptibility, thereby necessitating model adaptation.
  arXiv  Detail & Related papers  (2023-10-23T11:58:01Z)
• Counterfactual Explanations for Time Series Forecasting [14.03870816983583]
  We formulate the novel problem of counterfactual generation for time series forecasting, and propose an algorithm, called ForecastCF. ForecastCF solves the problem by applying gradient-based perturbations to the original time series. Our results show that ForecastCF outperforms the baseline in terms of counterfactual validity and data manifold closeness.
  arXiv  Detail & Related papers  (2023-10-12T08:51:59Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Sequential Predictive Conformal Inference for Time Series [16.38369532102931]
  We present a new distribution-free conformal prediction algorithm for sequential data (e.g., time series) We specifically account for the nature that time series data are non-exchangeable, and thus many existing conformal prediction algorithms are not applicable.
  arXiv  Detail & Related papers  (2022-12-07T05:07:27Z)
• Adaptive Conformal Predictions for Time Series [0.0]
  We argue that Adaptive Conformal Inference (ACI) is a good procedure for time series with general dependency. We propose a parameter-free method, AgACI, that adaptively builds upon ACI based on online expert aggregation. We conduct a real case study: electricity price forecasting.
  arXiv  Detail & Related papers  (2022-02-15T09:57:01Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.