Diverse Rule Sets

• URL: http://arxiv.org/abs/2006.09890v1
• Date: Wed, 17 Jun 2020 14:15:25 GMT
• Title: Diverse Rule Sets
• Authors: Guangyi Zhang and Aristides Gionis
• Abstract summary: Rule-based systems are experiencing a renaissance owing to their intuitive if-then representation. We propose a novel approach of inferring diverse rule sets, by optimizing small overlap among decision rules. We then devise an efficient randomized algorithm, which samples rules that are highly discriminative and have small overlap.
• Score: 20.170305081348328
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While machine-learning models are flourishing and transforming many aspects of everyday life, the inability of humans to understand complex models poses difficulties for these models to be fully trusted and embraced. Thus, interpretability of models has been recognized as an equally important quality as their predictive power. In particular, rule-based systems are experiencing a renaissance owing to their intuitive if-then representation. However, simply being rule-based does not ensure interpretability. For example, overlapped rules spawn ambiguity and hinder interpretation. Here we propose a novel approach of inferring diverse rule sets, by optimizing small overlap among decision rules with a 2-approximation guarantee under the framework of Max-Sum diversification. We formulate the problem as maximizing a weighted sum of discriminative quality and diversity of a rule set. In order to overcome an exponential-size search space of association rules, we investigate several natural options for a small candidate set of high-quality rules, including frequent and accurate rules, and examine their hardness. Leveraging the special structure in our formulation, we then devise an efficient randomized algorithm, which samples rules that are highly discriminative and have small overlap. The proposed sampling algorithm analytically targets a distribution of rules that is tailored to our objective. We demonstrate the superior predictive power and interpretability of our model with a comprehensive empirical study against strong baselines.

Related papers

• A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers [20.416696003269674]
  This paper introduces a scalable visual analysis method to explain tree ensemble classifiers that contain tens of thousands of rules. We develop an anomaly-biased model reduction method to prioritize these rules at each hierarchical level. Our method fosters a deeper understanding of both common and anomalous rules, thereby enhancing interpretability without sacrificing comprehensiveness.
  arXiv  Detail & Related papers  (2024-09-05T01:48:11Z)
• Probabilistic Truly Unordered Rule Sets [4.169915659794567]
  We propose TURS, for Truly Unordered Rule Sets. We exploit the probabilistic properties of our rule sets, with the intuition of only allowing rules to overlap if they have similar probabilistic outputs. We benchmark against a wide range of rule-based methods and demonstrate that our method learns rule sets that have lower model complexity and highly competitive predictive performance.
  arXiv  Detail & Related papers  (2024-01-18T12:03:19Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• On Regularization and Inference with Label Constraints [62.60903248392479]
  We compare two strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference. For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints. For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage.
  arXiv  Detail & Related papers  (2023-07-08T03:39:22Z)
• Learning Locally Interpretable Rule Ensemble [2.512827436728378]
  A rule ensemble is an interpretable model based on the linear combination of weighted rules. This paper proposes a new framework for learning a rule ensemble model that is both accurate and interpretable.
  arXiv  Detail & Related papers  (2023-06-20T12:06:56Z)
• Concise and interpretable multi-label rule sets [13.416159628299779]
  We develop a multi-label classifier that can be represented as a concise set of simple "if-then" rules. Our method is able to find a small set of relevant patterns that lead to accurate multi-label classification.
  arXiv  Detail & Related papers  (2022-10-04T11:23:50Z)
• Universal and data-adaptive algorithms for model selection in linear contextual bandits [52.47796554359261]
  We consider the simplest non-trivial instance of model-selection: distinguishing a simple multi-armed bandit problem from a linear contextual bandit problem. We introduce new algorithms that explore in a data-adaptive manner and provide guarantees of the form $mathcalO(dalpha T1- alpha)$. Our approach extends to model selection among nested linear contextual bandits under some additional assumptions.
  arXiv  Detail & Related papers  (2021-11-08T18:05:35Z)
• Preference learning along multiple criteria: A game-theoretic perspective [97.94912276610002]
  We generalize the notion of a von Neumann winner to the multi-criteria setting by taking inspiration from Blackwell's approachability. Our framework allows for non-linear aggregation of preferences across criteria, and generalizes the linearization-based approach from multi-objective optimization. We show that the Blackwell winner of a multi-criteria problem instance can be computed as the solution to a convex optimization problem.
  arXiv  Detail & Related papers  (2021-05-05T03:23:11Z)
• Characterizing Fairness Over the Set of Good Models Under Selective Labels [69.64662540443162]
  We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance. We provide tractable algorithms to compute the range of attainable group-level predictive disparities. We extend our framework to address the empirically relevant challenge of selectively labelled data.
  arXiv  Detail & Related papers  (2021-01-02T02:11:37Z)
• Squared $\ell_2$ Norm as Consistency Loss for Leveraging Augmented Data to Learn Robust and Invariant Representations [76.85274970052762]
  Regularizing distance between embeddings/representations of original samples and augmented counterparts is a popular technique for improving robustness of neural networks. In this paper, we explore these various regularization choices, seeking to provide a general understanding of how we should regularize the embeddings. We show that the generic approach we identified (squared $ell$ regularized augmentation) outperforms several recent methods, which are each specially designed for one task.
  arXiv  Detail & Related papers  (2020-11-25T22:40:09Z)
• Generation of Consistent Sets of Multi-Label Classification Rules with a Multi-Objective Evolutionary Algorithm [11.25469393912791]
  We propose a multi-objective evolutionary algorithm that generates multiple rule-based multi-label classification models. Our algorithm generates models based on sets (unordered collections) of rules, increasing interpretability. Also, by employing a conflict avoidance algorithm during the rule-creation, every rule within a given model is guaranteed to be consistent with every other rule in the same model.
  arXiv  Detail & Related papers  (2020-03-27T16:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.