On the benefits of maximum likelihood estimation for Regression and Forecasting

• URL: http://arxiv.org/abs/2106.10370v1
• Date: Fri, 18 Jun 2021 22:10:43 GMT
• Title: On the benefits of maximum likelihood estimation for Regression and Forecasting
• Authors: Pranjal Awasthi, Abhimanyu Das, Rajat Sen, Ananda Theertha Suresh
• Abstract summary: We advocate for a practical Maximum Likelihood Estimation (MLE) approach for regression and forecasting. This approach is better suited to capture inductive biases such as prior domain knowledge in datasets. We demonstrate empirically that our method instantiated with a well-designed general purpose mixture likelihood family can obtain superior performance over Empirical Risk Minimization.
• Score: 35.386189585135334
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We advocate for a practical Maximum Likelihood Estimation (MLE) approach for regression and forecasting, as an alternative to the typical approach of Empirical Risk Minimization (ERM) for a specific target metric. This approach is better suited to capture inductive biases such as prior domain knowledge in datasets, and can output post-hoc estimators at inference time that can optimize different types of target metrics. We present theoretical results to demonstrate that our approach is always competitive with any estimator for the target metric under some general conditions, and in many practical settings (such as Poisson Regression) can actually be much superior to ERM. We demonstrate empirically that our method instantiated with a well-designed general purpose mixture likelihood family can obtain superior performance over ERM for a variety of tasks across time-series forecasting and regression datasets with different data distributions.

Related papers

• Ranking and Combining Latent Structured Predictive Scores without Labeled Data [2.5064967708371553]
  This paper introduces a novel structured unsupervised ensemble learning model (SUEL) It exploits the dependency between a set of predictors with continuous predictive scores, rank the predictors without labeled data and combine them to an ensembled score with weights. The efficacy of the proposed methods is rigorously assessed through both simulation studies and real-world application of risk genes discovery.
  arXiv  Detail & Related papers  (2024-08-14T20:14:42Z)
• SEMF: Supervised Expectation-Maximization Framework for Predicting Intervals [0.8192907805418583]
  Supervised Expectation-Maximization Framework (SEMF) is a versatile and model-agnostic approach for generating prediction intervals in datasets with complete or missing data. SEMF extends the Expectation-Maximization algorithm, traditionally used in unsupervised learning, to a supervised context, leveraging latent variable modeling for uncertainty estimation.
  arXiv  Detail & Related papers  (2024-05-28T13:43:34Z)
• Regression-aware Inference with LLMs [52.764328080398805]
  We show that an inference strategy can be sub-optimal for common regression and scoring evaluation metrics. We propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses.
  arXiv  Detail & Related papers  (2024-03-07T03:24:34Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• Probable Domain Generalization via Quantile Risk Minimization [90.15831047587302]
  Domain generalization seeks predictors which perform well on unseen test distributions. We propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability.
  arXiv  Detail & Related papers  (2022-07-20T14:41:09Z)
• Statistical Learning for Individualized Asset Allocation [22.053470518472356]
  We develop a discretization approach to model the effect from continuous actions. We show that our estimators using generalized folded concave penalties enjoy desirable theoretical properties. The results show that our individualized optimal strategy improves individual financial well-being and surpasses benchmark strategies.
  arXiv  Detail & Related papers  (2022-01-20T04:40:03Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• An Extended Multi-Model Regression Approach for Compressive Strength Prediction and Optimization of a Concrete Mixture [0.0]
  A model based evaluation of concrete compressive strength is of high value, both for the purpose of strength prediction and the mixture optimization. We take a further step towards improving the accuracy of the prediction model via the weighted combination of multiple regression methods. A proposed (GA)-based mixture optimization is proposed, building on the obtained multi-regression model.
  arXiv  Detail & Related papers  (2021-06-13T16:10:32Z)
• Approximate Cross-validated Mean Estimates for Bayesian Hierarchical Regression Models [6.824747267214373]
  We introduce a novel procedure for obtaining cross-validated predictive estimates for Bayesian hierarchical regression models. We provide theoretical results and demonstrate its efficacy on publicly available data and in simulations.
  arXiv  Detail & Related papers  (2020-11-29T00:00:20Z)
• Combining Task Predictors via Enhancing Joint Predictability [53.46348489300652]
  We present a new predictor combination algorithm that improves the target by i) measuring the relevance of references based on their capabilities in predicting the target, and ii) strengthening such estimated relevance. Our algorithm jointly assesses the relevance of all references by adopting a Bayesian framework. Based on experiments on seven real-world datasets from visual attribute ranking and multi-class classification scenarios, we demonstrate that our algorithm offers a significant performance gain and broadens the application range of existing predictor combination approaches.
  arXiv  Detail & Related papers  (2020-07-15T21:58:39Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.