Efficient Exploration of the Rashomon Set of Rule Set Models

• URL: http://arxiv.org/abs/2406.03059v1
• Date: Wed, 5 Jun 2024 08:37:41 GMT
• Title: Efficient Exploration of the Rashomon Set of Rule Set Models
• Authors: Martino Ciaperoni, Han Xiao, Aristides Gionis,
• Abstract summary: An emerging paradigm in interpretable machine learning aims at exploring the Rashomon set of all models exhibiting near-optimal performance. Existing work on Rashomon-set exploration focuses on exhaustive search of the Rashomon set for particular classes of models. We propose, for the first time, efficient methods to explore the Rashomon set of rule set models with or without exhaustive search.
• Score: 18.187800166484507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Today, as increasingly complex predictive models are developed, simple rule sets remain a crucial tool to obtain interpretable predictions and drive high-stakes decision making. However, a single rule set provides a partial representation of a learning task. An emerging paradigm in interpretable machine learning aims at exploring the Rashomon set of all models exhibiting near-optimal performance. Existing work on Rashomon-set exploration focuses on exhaustive search of the Rashomon set for particular classes of models, which can be a computationally challenging task. On the other hand, exhaustive enumeration leads to redundancy that often is not necessary, and a representative sample or an estimate of the size of the Rashomon set is sufficient for many applications. In this work, we propose, for the first time, efficient methods to explore the Rashomon set of rule set models with or without exhaustive search. Extensive experiments demonstrate the effectiveness of the proposed methods in a variety of scenarios.

Related papers

• Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning [78.72226641279863]
  Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. Our research explores task-specific model pruning to inform decisions about designing SMoE architectures. We introduce an adaptive task-aware pruning technique UNCURL to reduce the number of experts per MoE layer in an offline manner post-training.
  arXiv  Detail & Related papers  (2024-09-02T22:35:03Z)
• Practical Attribution Guidance for Rashomon Sets [0.0]
  We study the problem of the Rashomon set sampling from a practical viewpoint. We identify two fundamental axioms - generalizability and implementation sparsity. We use the norms to guide the design of an $epsilon$-subgradient-based sampling method.
  arXiv  Detail & Related papers  (2024-07-26T03:17:41Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation [15.556756363296543]
  Predictive multiplicity refers to the phenomenon in which classification tasks admit multiple competing models that achieve almost-equally-optimal performance. We propose a novel framework that utilizes dropout techniques for exploring models in the Rashomon set. We show that our technique consistently outperforms baselines in terms of the effectiveness of predictive multiplicity metric estimation.
  arXiv  Detail & Related papers  (2024-02-01T16:25:00Z)
• A Path to Simpler Models Starts With Noise [17.36067410506525]
  The Rashomon set is the set of models that perform approximately equally well on a given dataset. An open question is why Rashomon ratios often tend to be large. We show that noisier datasets lead to larger Rashomon ratios through the way that practitioners train models.
  arXiv  Detail & Related papers  (2023-10-30T16:52:57Z)
• Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations [60.241144377865716]
  We consider the imitation of sub-optimal demonstrations, with both a small clean demonstration set and a large noisy set. We propose method by evaluating and imitating at the sub-demonstration level, encoding action primitives of varying quality into different skills.
  arXiv  Detail & Related papers  (2023-06-13T17:24:37Z)
• Exploring and Interacting with the Set of Good Sparse Generalized Additive Models [26.64299550434767]
  We present algorithms to approximate the Rashomon set of sparse, generalized additive models with ellipsoids for fixed support sets. The approximated Rashomon set serves as a cornerstone to solve practical challenges such as (1) studying the variable importance for the model class; (2) finding models under user-specified constraints (monotonicity, direct editing); and (3) investigating sudden changes in the shape functions.
  arXiv  Detail & Related papers  (2023-03-28T15:25:46Z)
• Exploring the Whole Rashomon Set of Sparse Decision Trees [23.136590456299007]
  We show that the Rashomon set is the set of all almost-optimal models. We provide the first technique for completely enumerating the Rashomon set for sparse decision trees. This allows the user an unprecedented level of control over model choice.
  arXiv  Detail & Related papers  (2022-09-16T16:37:26Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• Few-shot Classification via Adaptive Attention [93.06105498633492]
  We propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples. As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
  arXiv  Detail & Related papers  (2020-08-06T05:52:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.