Selecting Diverse Models for Scientific Insight

• URL: http://arxiv.org/abs/2006.09157v3
• Date: Thu, 16 Dec 2021 01:17:34 GMT
• Title: Selecting Diverse Models for Scientific Insight
• Authors: Laura J. Wendelberger, Brian J. Reich, Alyson G. Wilson
• Abstract summary: We show how different penalty settings can promote either shrinkage or sparsity of coefficients in separate models. A choice of penalty form that enforces variable selection is applied to predict stacking fault energy from steel alloy composition.
• Score: 0.12891210250935145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model selection often aims to choose a single model, assuming that the form of the model is correct. However, there may be multiple possible underlying explanatory patterns in a set of predictors that could explain a response. Model selection without regard for model uncertainty can fail to bring these patterns to light. We explore multi-model penalized regression (MMPR) to acknowledge model uncertainty in the context of penalized regression. We examine how different penalty settings can promote either shrinkage or sparsity of coefficients in separate models. The method is tuned to explicitly limit model similarity. A choice of penalty form that enforces variable selection is applied to predict stacking fault energy (SFE) from steel alloy composition. The aim is to identify multiple models with different subsets of covariates that explain a single type of response.

Related papers

• Stabilizing black-box model selection with the inflated argmax [8.52745154080651]
  This paper presents a new approach to stabilizing model selection that leverages a combination of bagging and an "inflated" argmax operation. Our method selects a small collection of models that all fit the data, and it is stable in that, with high probability, the removal of any training point will result in a collection of selected models that overlaps with the original collection. In both settings, the proposed method yields stable and compact collections of selected models, outperforming a variety of benchmarks.
  arXiv  Detail & Related papers  (2024-10-23T20:39:07Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers [66.36045164286854]
  We analyze a set of existing bias features and demonstrate there is no single model that works best for all the cases. By choosing an appropriate bias model, we can obtain a better robustness result than baselines with a more sophisticated model design.
  arXiv  Detail & Related papers  (2022-10-28T17:52:10Z)
• Bayesian Model Selection of Lithium-Ion Battery Models via Bayesian Quadrature [15.427086271923445]
  This paper presents a Bayesian model selection approach using Bayesian quadrature. The model evidence is adopted as the selection metric, choosing the simplest model that describes the data, in the spirit of Occam's razor. We show that popular model selection criteria, such as root-mean-square error and Bayesian information criterion, can fail to select a parsimonious model in the case of a multimodal posterior.
  arXiv  Detail & Related papers  (2022-10-28T15:24:17Z)
• Positive-Congruent Training: Towards Regression-Free Model Updates [87.25247195148187]
  In image classification, sample-wise inconsistencies appear as "negative flips" A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model.
  arXiv  Detail & Related papers  (2020-11-18T09:00:44Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Learning Invariances for Interpretability using Supervised VAE [0.0]
  We learn model invariances as a means of interpreting a model. We propose a supervised form of variational auto-encoders (VAEs) We show how combining our model with feature attribution methods it is possible to reach a more fine-grained understanding about the decision process of the model.
  arXiv  Detail & Related papers  (2020-07-15T10:14:16Z)
• Semi-nonparametric Latent Class Choice Model with a Flexible Class Membership Component: A Mixture Model Approach [6.509758931804479]
  The proposed model formulates the latent classes using mixture models as an alternative approach to the traditional random utility specification. Results show that mixture models improve the overall performance of latent class choice models.
  arXiv  Detail & Related papers  (2020-07-06T13:19:26Z)
• Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
  We show that a posterior approximation distinct from the variational distribution should be used for making decisions. Motivated by these theoretical results, we propose learning several approximate proposals for the best model. In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
  arXiv  Detail & Related papers  (2020-02-17T19:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.