Amazing Things Come From Having Many Good Models

• URL: http://arxiv.org/abs/2407.04846v2
• Date: Wed, 10 Jul 2024 02:39:01 GMT
• Title: Amazing Things Come From Having Many Good Models
• Authors: Cynthia Rudin, Chudi Zhong, Lesia Semenova, Margo Seltzer, Ronald Parr, Jiachang Liu, Srikar Katta, Jon Donnelly, Harry Chen, Zachery Boner,
• Abstract summary: The Rashomon Effect describes the phenomenon that there exist many equally good predictive models for the same dataset. This perspective piece proposes reshaping the way we think about machine learning. Our goal is to illustrate how the Rashomon Effect can have a massive impact on the use of machine learning for complex problems in society.
• Score: 15.832860655980918
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The Rashomon Effect, coined by Leo Breiman, describes the phenomenon that there exist many equally good predictive models for the same dataset. This phenomenon happens for many real datasets and when it does, it sparks both magic and consternation, but mostly magic. In light of the Rashomon Effect, this perspective piece proposes reshaping the way we think about machine learning, particularly for tabular data problems in the nondeterministic (noisy) setting. We address how the Rashomon Effect impacts (1) the existence of simple-yet-accurate models, (2) flexibility to address user preferences, such as fairness and monotonicity, without losing performance, (3) uncertainty in predictions, fairness, and explanations, (4) reliable variable importance, (5) algorithm choice, specifically, providing advanced knowledge of which algorithms might be suitable for a given problem, and (6) public policy. We also discuss a theory of when the Rashomon Effect occurs and why. Our goal is to illustrate how the Rashomon Effect can have a massive impact on the use of machine learning for complex problems in society.

Related papers

• A Path to Simpler Models Starts With Noise [17.36067410506525]
  The Rashomon set is the set of models that perform approximately equally well on a given dataset. An open question is why Rashomon ratios often tend to be large. We show that noisier datasets lead to larger Rashomon ratios through the way that practitioners train models.
  arXiv  Detail & Related papers  (2023-10-30T16:52:57Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning [2.60815298676312]
  The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics.
  arXiv  Detail & Related papers  (2023-06-27T20:32:07Z)
• Non-Invasive Fairness in Learning through the Lens of Data Drift [88.37640805363317]
  We show how to improve the fairness of Machine Learning models without altering the data or the learning algorithm. We use a simple but key insight: the divergence of trends between different populations, and, consecutively, between a learned model and minority populations, is analogous to data drift. We explore two strategies (model-splitting and reweighing) to resolve this drift, aiming to improve the overall conformance of models to the underlying data.
  arXiv  Detail & Related papers  (2023-03-30T17:30:42Z)
• CLEAR: Generative Counterfactual Explanations on Graphs [60.30009215290265]
  We study the problem of counterfactual explanation generation on graphs. A few studies have explored counterfactual explanations on graphs, but many challenges of this problem are still not well-addressed. We propose a novel framework CLEAR which aims to generate counterfactual explanations on graphs for graph-level prediction models.
  arXiv  Detail & Related papers  (2022-10-16T04:35:32Z)
• Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability [0.0]
  We study tradeoff between model performance and explainability of machine learning algorithms. We find that the tradeoff is much less gradual in the end user's perception. Results of our second experiment show that while explainable artificial intelligence augmentations can be used to increase explainability, the type of explanation plays an essential role in end user perception.
  arXiv  Detail & Related papers  (2022-06-20T08:32:38Z)
• Counterfactual Fairness with Partially Known Causal Graph [85.15766086381352]
  This paper proposes a general method to achieve the notion of counterfactual fairness when the true causal graph is unknown. We find that counterfactual fairness can be achieved as if the true causal graph were fully known, when specific background knowledge is provided.
  arXiv  Detail & Related papers  (2022-05-27T13:40:50Z)
• A Reflection on Learning from Data: Epistemology Issues and Limitations [1.8047694351309205]
  This paper reflects on some issues and some limitations of the knowledge discovered in data. The paper sheds some light on the shortcomings of using generic mathematical theories to describe the process. It further highlights the need for theories specialized in learning from data.
  arXiv  Detail & Related papers  (2021-07-28T11:05:34Z)
• Revisiting Rashomon: A Comment on "The Two Cultures" [95.81740983484471]
  Breiman dubbed the "Rashomon Effect", describing the situation in which there are many models that satisfy predictive accuracy criteria equally well, but process information in substantially different ways. This phenomenon can make it difficult to draw conclusions or automate decisions based on a model fit to data. I make connections to recent work in the Machine Learning literature that explore the implications of this issue.
  arXiv  Detail & Related papers  (2021-04-05T20:51:58Z)
• An Information-Theoretic Approach to Personalized Explainable Machine Learning [92.53970625312665]
  We propose a simple probabilistic model for the predictions and user knowledge. We quantify the effect of an explanation by the conditional mutual information between the explanation and prediction.
  arXiv  Detail & Related papers  (2020-03-01T13:06:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.