Dynamic Inference
- URL: http://arxiv.org/abs/2111.14746v2
- Date: Tue, 30 Nov 2021 18:33:53 GMT
- Title: Dynamic Inference
- Authors: Aolin Xu
- Abstract summary: In some sequential estimation problems, the future values of the quantity to be estimated depend on the estimate of its current value.
Examples include stock price prediction by big investors, interactive product recommendation, and behavior prediction in multi-agent systems.
In this work, a formulation of this problem under a Bayesian probabilistic framework is given, and the optimal estimation strategy is derived as the solution to minimize the overall inference loss.
- Score: 4.568777157687959
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Traditional statistical estimation, or statistical inference in general, is
static, in the sense that the estimate of the quantity of interest does not
change the future evolution of the quantity. In some sequential estimation
problems however, we encounter the situation where the future values of the
quantity to be estimated depend on the estimate of its current value. Examples
include stock price prediction by big investors, interactive product
recommendation, and behavior prediction in multi-agent systems. We may call
such problems as dynamic inference. In this work, a formulation of this problem
under a Bayesian probabilistic framework is given, and the optimal estimation
strategy is derived as the solution to minimize the overall inference loss. How
the optimal estimation strategy works is illustrated through two examples,
stock trend prediction and vehicle behavior prediction. When the underlying
models for dynamic inference are unknown, we can consider the problem of
learning for dynamic inference. This learning problem can potentially unify
several familiar machine learning problems, including supervised learning,
imitation learning, and reinforcement learning.
Related papers
- Evaluating Pedestrian Trajectory Prediction Methods with Respect to Autonomous Driving [0.9217021281095907]
In this paper, we assess the state of the art in pedestrian trajectory prediction within the context of generating single trajectories.
The evaluation is conducted on the widely-used ETH/UCY dataset where the Average Displacement Error (ADE) and the Final Displacement Error (FDE) are reported.
arXiv Detail & Related papers (2023-08-09T19:21:50Z) - Bayesian Learning for Dynamic Inference [2.2843885788439793]
In some sequential estimation problems, the future values of the quantity to be estimated depend on the estimate of its current value.
We formulate the Bayesian learning problem for dynamic inference, where the unknown quantity-generation model is assumed to be randomly drawn.
We derive the optimal Bayesian learning rules, both offline and online, to minimize the inference loss.
arXiv Detail & Related papers (2022-12-30T19:16:23Z) - Towards Out-of-Distribution Sequential Event Prediction: A Causal
Treatment [72.50906475214457]
The goal of sequential event prediction is to estimate the next event based on a sequence of historical events.
In practice, the next-event prediction models are trained with sequential data collected at one time.
We propose a framework with hierarchical branching structures for learning context-specific representations.
arXiv Detail & Related papers (2022-10-24T07:54:13Z) - Prediction of Dilatory Behavior in eLearning: A Comparison of Multiple
Machine Learning Models [0.2963240482383777]
Procrastination, the irrational delay of tasks, is a common occurrence in online learning.
Research focusing on such predictions is scarce.
Studies involving different types of predictors and comparisons between the predictive performance of various methods are virtually non-existent.
arXiv Detail & Related papers (2022-06-30T07:24:08Z) - Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
We study off-policy evaluation and learning from sequential data.
We formalize the relevant causal structure of problems such as dynamic personalized pricing.
We show improved out-of-sample policy performance in this class of relevant problems.
arXiv Detail & Related papers (2021-10-19T16:15:56Z) - Unifying Gradient Estimators for Meta-Reinforcement Learning via
Off-Policy Evaluation [53.83642844626703]
We provide a unifying framework for estimating higher-order derivatives of value functions, based on off-policy evaluation.
Our framework interprets a number of prior approaches as special cases and elucidates the bias and variance trade-off of Hessian estimates.
arXiv Detail & Related papers (2021-06-24T15:58:01Z) - Foresee then Evaluate: Decomposing Value Estimation with Latent Future
Prediction [37.06232589005015]
Value function is the central notion of Reinforcement Learning (RL)
We propose Value Decomposition with Future Prediction (VDFP)
We analytically decompose the value function into a latent future dynamics part and a policy-independent trajectory return part, inducing a way to model latent dynamics and returns separately in value estimation.
arXiv Detail & Related papers (2021-03-03T07:28:56Z) - Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
We propose a novel scalable method to learn double-robust representations for counterfactual predictions.
We make robust and efficient counterfactual predictions for both individual and average treatment effects.
The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
arXiv Detail & Related papers (2020-10-15T16:39:26Z) - Counterfactual Predictions under Runtime Confounding [74.90756694584839]
We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data.
We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
arXiv Detail & Related papers (2020-06-30T15:49:05Z) - Value-driven Hindsight Modelling [68.658900923595]
Value estimation is a critical component of the reinforcement learning (RL) paradigm.
Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function.
We develop an approach for representation learning in RL that sits in between these two extremes.
This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function.
arXiv Detail & Related papers (2020-02-19T18:10:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.