Exploring and Interacting with the Set of Good Sparse Generalized Additive Models

• URL: http://arxiv.org/abs/2303.16047v3
• Date: Fri, 17 Nov 2023 15:41:33 GMT
• Title: Exploring and Interacting with the Set of Good Sparse Generalized Additive Models
• Authors: Chudi Zhong, Zhi Chen, Jiachang Liu, Margo Seltzer, Cynthia Rudin
• Abstract summary: We present algorithms to approximate the Rashomon set of sparse, generalized additive models with ellipsoids for fixed support sets. The approximated Rashomon set serves as a cornerstone to solve practical challenges such as (1) studying the variable importance for the model class; (2) finding models under user-specified constraints (monotonicity, direct editing); and (3) investigating sudden changes in the shape functions.
• Score: 26.64299550434767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In real applications, interaction between machine learning models and domain experts is critical; however, the classical machine learning paradigm that usually produces only a single model does not facilitate such interaction. Approximating and exploring the Rashomon set, i.e., the set of all near-optimal models, addresses this practical challenge by providing the user with a searchable space containing a diverse set of models from which domain experts can choose. We present algorithms to efficiently and accurately approximate the Rashomon set of sparse, generalized additive models with ellipsoids for fixed support sets and use these ellipsoids to approximate Rashomon sets for many different support sets. The approximated Rashomon set serves as a cornerstone to solve practical challenges such as (1) studying the variable importance for the model class; (2) finding models under user-specified constraints (monotonicity, direct editing); and (3) investigating sudden changes in the shape functions. Experiments demonstrate the fidelity of the approximated Rashomon set and its effectiveness in solving practical challenges.

Related papers

• Dynamic Post-Hoc Neural Ensemblers [55.15643209328513]
  In this study, we explore employing neural networks as ensemble methods. Motivated by the risk of learning low-diversity ensembles, we propose regularizing the model by randomly dropping base model predictions. We demonstrate this approach lower bounds the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Efficient Exploration of the Rashomon Set of Rule Set Models [18.187800166484507]
  An emerging paradigm in interpretable machine learning aims at exploring the Rashomon set of all models exhibiting near-optimal performance. Existing work on Rashomon-set exploration focuses on exhaustive search of the Rashomon set for particular classes of models. We propose, for the first time, efficient methods to explore the Rashomon set of rule set models with or without exhaustive search.
  arXiv  Detail & Related papers  (2024-06-05T08:37:41Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation [15.556756363296543]
  Predictive multiplicity refers to the phenomenon in which classification tasks admit multiple competing models that achieve almost-equally-optimal performance. We propose a novel framework that utilizes dropout techniques for exploring models in the Rashomon set. We show that our technique consistently outperforms baselines in terms of the effectiveness of predictive multiplicity metric estimation.
  arXiv  Detail & Related papers  (2024-02-01T16:25:00Z)
• A Path to Simpler Models Starts With Noise [17.36067410506525]
  The Rashomon set is the set of models that perform approximately equally well on a given dataset. An open question is why Rashomon ratios often tend to be large. We show that noisier datasets lead to larger Rashomon ratios through the way that practitioners train models.
  arXiv  Detail & Related papers  (2023-10-30T16:52:57Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)
• Exploring the Whole Rashomon Set of Sparse Decision Trees [23.136590456299007]
  We show that the Rashomon set is the set of all almost-optimal models. We provide the first technique for completely enumerating the Rashomon set for sparse decision trees. This allows the user an unprecedented level of control over model choice.
  arXiv  Detail & Related papers  (2022-09-16T16:37:26Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• Data Summarization via Bilevel Optimization [48.89977988203108]
  A simple yet powerful approach is to operate on small subsets of data. In this work, we propose a generic coreset framework that formulates the coreset selection as a cardinality-constrained bilevel optimization problem.
  arXiv  Detail & Related papers  (2021-09-26T09:08:38Z)
• Conditional Generative Modeling via Learning the Latent Space [54.620761775441046]
  We propose a novel framework for conditional generation in multimodal spaces. It uses latent variables to model generalizable learning patterns. At inference, the latent variables are optimized to find optimal solutions corresponding to multiple output modes.
  arXiv  Detail & Related papers  (2020-10-07T03:11:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.