SWAG: A Wrapper Method for Sparse Learning

• URL: http://arxiv.org/abs/2006.12837v2
• Date: Mon, 1 Nov 2021 01:12:06 GMT
• Title: SWAG: A Wrapper Method for Sparse Learning
• Authors: Roberto Molinari, Gaetan Bakalli, St\'ephane Guerrier, Cesare Miglioli, Samuel Orso, Mucyo Karemera, Olivier Scaillet
• Abstract summary: We propose a procedure to find a library of sparse learners with consequent low data collection and storage costs. This new method delivers a low-dimensional network of attributes that can be easily interpreted. We call this algorithm "Sparse Wrapper AlGorithm" (SWAG)
• Score: 0.13854111346209866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The majority of machine learning methods and algorithms give high priority to prediction performance which may not always correspond to the priority of the users. In many cases, practitioners and researchers in different fields, going from engineering to genetics, require interpretability and replicability of the results especially in settings where, for example, not all attributes may be available to them. As a consequence, there is the need to make the outputs of machine learning algorithms more interpretable and to deliver a library of "equivalent" learners (in terms of prediction performance) that users can select based on attribute availability in order to test and/or make use of these learners for predictive/diagnostic purposes. To address these needs, we propose to study a procedure that combines screening and wrapper approaches which, based on a user-specified learning method, greedily explores the attribute space to find a library of sparse learners with consequent low data collection and storage costs. This new method (i) delivers a low-dimensional network of attributes that can be easily interpreted and (ii) increases the potential replicability of results based on the diversity of attribute combinations defining strong learners with equivalent predictive power. We call this algorithm "Sparse Wrapper AlGorithm" (SWAG).

Related papers

• LLM-Select: Feature Selection with Large Language Models [64.5099482021597]
  Large language models (LLMs) are capable of selecting the most predictive features, with performance rivaling the standard tools of data science. Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place.
  arXiv  Detail & Related papers  (2024-07-02T22:23:40Z)
• Algorithmic failure as a humanities methodology: machine learning's mispredictions identify rich cases for qualitative analysis [0.0]
  I trained a simple machine learning algorithm to predict whether or not an action was active or passive using only information about fictional characters. The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis. Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative.
  arXiv  Detail & Related papers  (2023-05-19T13:24:32Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• The Information Geometry of Unsupervised Reinforcement Learning [133.20816939521941]
  Unsupervised skill discovery is a class of algorithms that learn a set of policies without access to a reward function. We show that unsupervised skill discovery algorithms do not learn skills that are optimal for every possible reward function.
  arXiv  Detail & Related papers  (2021-10-06T13:08:36Z)
• Generative and reproducible benchmarks for comprehensive evaluation of machine learning classifiers [6.605210393590192]
  DIverse and GENerative ML Benchmark (DIGEN) is a collection of synthetic datasets for benchmarking of machine learning algorithms. The resource with extensive documentation and analyses is open-source and available on GitHub.
  arXiv  Detail & Related papers  (2021-07-14T03:58:02Z)
• Combining Feature and Instance Attribution to Detect Artifacts [62.63504976810927]
  We propose methods to facilitate identification of training data artifacts. We show that this proposed training-feature attribution approach can be used to uncover artifacts in training data. We execute a small user study to evaluate whether these methods are useful to NLP researchers in practice.
  arXiv  Detail & Related papers  (2021-07-01T09:26:13Z)
• Discriminative, Generative and Self-Supervised Approaches for Target-Agnostic Learning [8.666667951130892]
  generative and self-supervised learning models are shown to perform well at the task. Our derived theorem for the pseudo-likelihood theory also shows that they are related for inferring a joint distribution model.
  arXiv  Detail & Related papers  (2020-11-12T15:03:40Z)
• Discovering Reinforcement Learning Algorithms [53.72358280495428]
  Reinforcement learning algorithms update an agent's parameters according to one of several possible rules. This paper introduces a new meta-learning approach that discovers an entire update rule. It includes both 'what to predict' (e.g. value functions) and 'how to learn from it' by interacting with a set of environments.
  arXiv  Detail & Related papers  (2020-07-17T07:38:39Z)
• Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
  Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method. PCL implicitly encodes semantic structures of the data into the learned embedding space. PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
  arXiv  Detail & Related papers  (2020-05-11T09:53:36Z)
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics [4.237343083490243]
  In machine learning (ML), ensemble methods such as bagging, boosting, and stacking are widely-established approaches. StackGenVis is a visual analytics system for stacked generalization.
  arXiv  Detail & Related papers  (2020-05-04T15:43:55Z)
• Learning to Ask Medical Questions using Reinforcement Learning [9.376814468000955]
  A reinforcement learning agent iteratively selects certain features to be unmasked, and uses them to predict an outcome when it is sufficiently confident. A key component of our approach is a guesser network, trained to predict the outcome from the selected features and parametrizing the reward function. Applying our method to a national survey dataset, we show that it not only outperforms strong baselines when requiring the prediction to be made based on a small number of input features, but is also highly more interpretable.
  arXiv  Detail & Related papers  (2020-03-31T18:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.