Tracking Changing Probabilities via Dynamic Learners

• URL: http://arxiv.org/abs/2402.10142v2
• Date: Tue, 30 Apr 2024 04:15:24 GMT
• Title: Tracking Changing Probabilities via Dynamic Learners
• Authors: Omid Madani,
• Abstract summary: We develop sparse multiclass moving average techniques to respond to non-stationarities in a timely manner. One technique is based on the exponentiated moving average (EMA) and another is based on queuing a few count snapshots.
• Score: 0.18648070031379424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Consider a predictor, a learner, whose input is a stream of discrete items. The predictor's task, at every time point, is probabilistic multiclass prediction, i.e., to predict which item may occur next by outputting zero or more candidate items, each with a probability, after which the actual item is revealed and the predictor learns from this observation. To output probabilities, the predictor keeps track of the proportions of the items it has seen. The stream is unbounded and the predictor has finite limited space and we seek efficient prediction and update techniques: the set of items is unknown to the predictor and their totality can also grow unbounded. Moreover, there is non-stationarity: the underlying frequencies of items may change, substantially, from time to time. For instance, new items may start appearing and a few recently frequent items may cease to occur again. The predictor, being space-bounded, need only provide probabilities for those items with (currently) sufficiently high frequency, i.e., the salient items. This problem is motivated in the setting of prediction games, a self-supervised learning regime where concepts serve as both the predictors and the predictands, and the set of concepts grows over time, resulting in non-stationarities as new concepts are generated and used. We develop sparse multiclass moving average techniques designed to respond to such non-stationarities in a timely manner. One technique is based on the exponentiated moving average (EMA) and another is based on queuing a few count snapshots. We show that the combination, and in particular supporting dynamic predictand-specific learning rates, offers advantages in terms of faster change detection and convergence.

Related papers

• Conformalized Late Fusion Multi-View Learning [18.928543069018865]
  Uncertainty quantification for multi-view learning is motivated by the increasing use of multi-view data in scientific problems. A common variant of multi-view learning is late fusion: train separate predictors on individual views and combine them after single-view predictions are available. We propose a novel methodology, Multi-View Conformal Prediction (MVCP), where conformal prediction is instead performed separately on the single-view predictors and only fused subsequently.
  arXiv  Detail & Related papers  (2024-05-25T14:11:01Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Sinkhorn-Flow: Predicting Probability Mass Flow in Dynamical Systems Using Optimal Transport [89.61692654941106]
  We propose a new approach to predicting such mass flow over time using optimal transport. We apply our approach to the task of predicting how communities will evolve over time in social network settings.
  arXiv  Detail & Related papers  (2023-03-14T07:25:44Z)
• Finding Islands of Predictability in Action Forecasting [7.215559809521136]
  We show that future action sequences are more accurately modeled with variable, rather than one, levels of abstraction. We propose a combination Bayesian neural network and hierarchical convolutional segmentation model to both accurately predict future actions and optimally select abstraction levels.
  arXiv  Detail & Related papers  (2022-10-13T21:01:16Z)
• Enhancing Stochastic Petri Net-based Remaining Time Prediction using k-Nearest Neighbors [0.5287304201523223]
  We extend remaining time prediction based on Petri nets with generally distributed transitions with k-nearest neighbors. The k-nearest neighbors algorithm is performed on simple storing the time passed to complete previous activities. We discuss the technique and its basic implementation in Python and use different real world data sets to evaluate the predictive power of our extension.
  arXiv  Detail & Related papers  (2022-06-27T08:27:35Z)
• Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression [51.770998056563094]
  Probabilistic Gradient Boosting Machines (PGBM) is a method to create probabilistic predictions with a single ensemble of decision trees. We empirically demonstrate the advantages of PGBM compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-03T08:32:13Z)
• Predicting MOOCs Dropout Using Only Two Easily Obtainable Features from the First Week's Activities [56.1344233010643]
  Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. This study aims to predict dropout early-on, from the first week, by comparing several machine-learning approaches.
  arXiv  Detail & Related papers  (2020-08-12T10:44:49Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.