A Method for Controlling Extrapolation when Visualizing and Optimizing the Prediction Profiles of Statistical and Machine Learning Models

• URL: http://arxiv.org/abs/2201.05236v1
• Date: Thu, 13 Jan 2022 22:31:32 GMT
• Title: A Method for Controlling Extrapolation when Visualizing and Optimizing the Prediction Profiles of Statistical and Machine Learning Models
• Authors: Jeremy Ash, Laura Lancaster, Chris Gotwalt
• Abstract summary: We present a novel method for controlling extrapolation in the prediction profiler in the JMP software. Our method helps users avoid exploring predictions that should be considered extrapolation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method for controlling extrapolation in the prediction profiler in the JMP software. The prediction profiler is a graphical tool for exploring high dimensional prediction surfaces for statistical and machine learning models. The profiler contains interactive cross-sectional views, or profile traces, of the prediction surface of a model. Our method helps users avoid exploring predictions that should be considered extrapolation. It also performs optimization over a constrained factor region that avoids extrapolation using a genetic algorithm. In simulations and real world examples, we demonstrate how optimal factor settings without constraint in the profiler are frequently extrapolated, and how extrapolation control helps avoid these solutions with invalid factor settings that may not be useful to the user.

Related papers

• Conformal Prediction for Image Segmentation Using Morphological Prediction Sets [0.7673339435080445]
  In this paper we focus on binary segmentation and address these challenges using conformal prediction. Our approach involves computing nonconformity scores, a type of prediction residual, on held-out calibration data not used during training. We evaluate our approach on several medical imaging applications.
  arXiv  Detail & Related papers  (2025-03-07T17:42:30Z)
• Critical Example Mining for Vehicle Trajectory Prediction using Flow-based Generative Models [10.40439055916036]
  This paper proposes a data-driven approach to estimate the rareness of the trajectories. By combining the rareness estimation of observations with whole trajectories, the proposed method effectively identifies a subset of data that is relatively hard to predict.
  arXiv  Detail & Related papers  (2024-10-21T15:02:30Z)
• Influence Functions for Scalable Data Attribution in Diffusion Models [52.92223039302037]
  Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges by developing an textitinfluence functions framework.
  arXiv  Detail & Related papers  (2024-10-17T17:59:02Z)
• Learning Augmentation Policies from A Model Zoo for Time Series Forecasting [58.66211334969299]
  We introduce AutoTSAug, a learnable data augmentation method based on reinforcement learning. By augmenting the marginal samples with a learnable policy, AutoTSAug substantially improves forecasting performance.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• Multi-Modal Conformal Prediction Regions with Simple Structures by Optimizing Convex Shape Templates [19.504348671777006]
  Conformal prediction is a statistical tool for producing prediction regions for machine learning models that are valid with high probability. A key component of conformal prediction algorithms is a emphnon-conformity score function that quantifies how different a model's prediction is from the unknown ground truth value. We propose a method that optimize parameterized emphshape template functions over calibration data, which results in non-conformity score functions that produce prediction regions with minimum volume.
  arXiv  Detail & Related papers  (2023-12-12T17:00:13Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• Conformal Prediction Regions for Time Series using Linear Complementarity Programming [25.094249285804224]
  We propose an optimization-based method for reducing conservatism to enable long horizon planning and verification. We show that this problem can be cast as a mixed integer linear complementarity program (MILCP), which we then relax into a linear complementarity program (LCP)
  arXiv  Detail & Related papers  (2023-04-03T15:32:38Z)
• Model Predictive Control with Gaussian-Process-Supported Dynamical Constraints for Autonomous Vehicles [82.65261980827594]
  We propose a model predictive control approach for autonomous vehicles that exploits learned Gaussian processes for predicting human driving behavior. A multi-mode predictive control approach considers the possible intentions of the human drivers.
  arXiv  Detail & Related papers  (2023-03-08T17:14:57Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• On Predictive Explanation of Data Anomalies [3.1798318618973362]
  PROTEUS is an AutoML pipeline designed for feature selection on imbalanced datasets. It produces predictive explanations by approximating the decision surface of an unsupervised detector. It reliably estimates their predictive performance in unseen data.
  arXiv  Detail & Related papers  (2021-10-18T16:59:28Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)
• Dimensionality reduction to maximize prediction generalization capability [0.7614628596146599]
  Generalization of time series prediction remains an important open issue in machine learning, wherein earlier methods have either large generalization error or local minima. We develop an analytically solvable, unsupervised learning scheme that extracts the most informative components for predicting future inputs, termed predictive principal component analysis (PredPCA) Our scheme can effectively remove unpredictable noise and minimize test prediction error through convex optimization.
  arXiv  Detail & Related papers  (2020-03-01T12:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.