Bayesian Quantile Regression with Subset Selection: A Decision Analysis Perspective

• URL: http://arxiv.org/abs/2311.02043v4
• Date: Sat, 16 Nov 2024 19:01:43 GMT
• Title: Bayesian Quantile Regression with Subset Selection: A Decision Analysis Perspective
• Authors: Joseph Feldman, Daniel Kowal,
• Abstract summary: Quantile regression is a powerful tool for inferring how covariates affect specific percentiles of the response distribution. Existing methods estimate conditional quantiles separately for each quantile of interest or estimate the entire conditional distribution using semi- or non-parametric models. We pose the fundamental problems of linear quantile estimation, uncertainty quantification, and subset selection from a Bayesian decision analysis perspective.
• Score: 0.0
• License:
• Abstract: Quantile regression is a powerful tool for inferring how covariates affect specific percentiles of the response distribution. Existing methods either estimate conditional quantiles separately for each quantile of interest or estimate the entire conditional distribution using semi- or non-parametric models. The former often produce inadequate models for real data and do not share information across quantiles, while the latter are characterized by complex and constrained models that can be difficult to interpret and computationally inefficient. Neither approach is well-suited for quantile-specific subset selection. Instead, we pose the fundamental problems of linear quantile estimation, uncertainty quantification, and subset selection from a Bayesian decision analysis perspective. For any Bayesian regression model -- including, but not limited to existing Bayesian quantile regression models -- we derive optimal point estimates, interpretable uncertainty quantification, and scalable subset selection techniques for all model-based conditional quantiles. Our approach introduces a quantile-focused squared error loss that enables efficient, closed-form computing and maintains a close relationship with Wasserstein-based density estimation. In an extensive simulation study, our methods demonstrate substantial gains in quantile estimation accuracy, inference, and variable selection over frequentist and Bayesian competitors. We use these tools to identify and quantify the heterogeneous impacts of multiple social stressors and environmental exposures on educational outcomes across the full spectrum of low-, medium-, and high-achieving students in North Carolina.

Related papers

• Semiparametric conformal prediction [79.6147286161434]
  Risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables. We treat the scores as random vectors and aim to construct the prediction set accounting for their joint correlation structure. We report desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs. We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint. We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Deep Non-Crossing Quantiles through the Partial Derivative [0.6299766708197883]
  Quantile Regression provides a way to approximate a single conditional quantile. Minimisation of the QR-loss function does not guarantee non-crossing quantiles. We propose a generic deep learning algorithm for predicting an arbitrary number of quantiles.
  arXiv  Detail & Related papers  (2022-01-30T15:35:21Z)
• Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting [12.269597033369557]
  We propose the Incremental (Spline) Quantile Functions I(S)QF, a flexible and efficient distribution-free quantile estimation framework. We also provide a generalization error analysis of our proposed approaches under the sequence-to-sequence setting.
  arXiv  Detail & Related papers  (2021-11-12T06:54:48Z)
• Communication-Efficient Distributed Quantile Regression with Optimal Statistical Guarantees [2.064612766965483]
  We address the problem of how to achieve optimal inference in distributed quantile regression without stringent scaling conditions. The difficulties are resolved through a double-smoothing approach that is applied to the local (at each data source) and global objective functions. Despite the reliance on a delicate combination of local and global smoothing parameters, the quantile regression model is fully parametric.
  arXiv  Detail & Related papers  (2021-10-25T17:09:59Z)
• Flexible Model Aggregation for Quantile Regression [92.63075261170302]
  Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions. We investigate methods for aggregating any number of conditional quantile models. All of the models we consider in this paper can be fit using modern deep learning toolkits.
  arXiv  Detail & Related papers  (2021-02-26T23:21:16Z)
• Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification [15.94100899123465]
  A model that predicts the true conditional quantiles for each input, at all quantile levels, presents a correct and efficient representation of the underlying uncertainty. Current quantile-based methods focus on optimizing the so-called pinball loss. We develop new quantile methods that address these shortcomings.
  arXiv  Detail & Related papers  (2020-11-18T23:51:23Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.