A Confidence Machine for Sparse High-Order Interaction Model

• URL: http://arxiv.org/abs/2205.14317v1
• Date: Sat, 28 May 2022 03:23:56 GMT
• Title: A Confidence Machine for Sparse High-Order Interaction Model
• Authors: Diptesh Das, Eugene Ndiaye and Ichiro Takeuchi
• Abstract summary: Conformal prediction (CP) is a promising approach for obtaining the confidence of prediction results with fewer theoretical assumptions. We develop a full-CP of sparse high-order interaction model (SHIM) which is sufficiently flexible as it can take into account high-order interactions among variables.
• Score: 16.780058676633914
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In predictive modeling for high-stake decision-making, predictors must be not only accurate but also reliable. Conformal prediction (CP) is a promising approach for obtaining the confidence of prediction results with fewer theoretical assumptions. To obtain the confidence set by so-called full-CP, we need to refit the predictor for all possible values of prediction results, which is only possible for simple predictors. For complex predictors such as random forests (RFs) or neural networks (NNs), split-CP is often employed where the data is split into two parts: one part for fitting and another to compute the confidence set. Unfortunately, because of the reduced sample size, split-CP is inferior to full-CP both in fitting as well as confidence set computation. In this paper, we develop a full-CP of sparse high-order interaction model (SHIM), which is sufficiently flexible as it can take into account high-order interactions among variables. We resolve the computational challenge for full-CP of SHIM by introducing a novel approach called homotopy mining. Through numerical experiments, we demonstrate that SHIM is as accurate as complex predictors such as RF and NN and enjoys the superior statistical power of full-CP.

Related papers

• Optimizing Probabilistic Conformal Prediction with Vectorized Non-Conformity Scores [6.059745771017814]
  We propose a novel framework that enhances efficiency by first vectorizing the non-conformity scores with ranked samples and then optimizing the shape of the prediction set by varying the quantiles for samples at the same rank. Our method delivers valid coverage while producing discontinuous and more efficient prediction sets, making it particularly suited for high-stakes applications.
  arXiv  Detail & Related papers  (2024-10-17T16:37:03Z)
• RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classification [24.52922162675259]
  Conformal prediction (CP) generates a set of predictions for a given test sample. The size of the set indicates how certain the predictions are. We propose a new method called Reliable-Region-Based Conformal Prediction (RR-CP)
  arXiv  Detail & Related papers  (2023-09-09T11:14:04Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Probabilistically robust conformal prediction [9.401004747930974]
  Conformal prediction (CP) is a framework to quantify uncertainty of machine learning classifiers including deep neural networks. Almost all the existing work on CP assumes clean testing data and there is not much known about the robustness of CP algorithms. This paper studies the problem of probabilistically robust conformal prediction (PRCP) which ensures robustness to most perturbations.
  arXiv  Detail & Related papers  (2023-07-31T01:32:06Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Improving Uncertainty Quantification of Deep Classifiers via Neighborhood Conformal Prediction: Novel Algorithm and Theoretical Analysis [30.0231328500976]
  Conformal prediction (CP) is a principled framework for uncertainty quantification of deep models. This paper proposes a novel algorithm referred to as Neighborhood Conformal Prediction (NCP) to improve the efficiency of uncertainty quantification. We show that NCP leads to significant reduction in prediction set size over prior CP methods.
  arXiv  Detail & Related papers  (2023-03-19T15:56:50Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• A novel Deep Learning approach for one-step Conformal Prediction approximation [0.7646713951724009]
  Conformal Prediction (CP) is a versatile solution that guarantees a maximum error rate given minimal constraints. We propose a novel conformal loss function that approximates the traditionally two-step CP approach in a single step.
  arXiv  Detail & Related papers  (2022-07-25T17:46:09Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.