A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning

• URL: http://arxiv.org/abs/2108.00490v1
• Date: Sun, 1 Aug 2021 16:47:15 GMT
• Title: A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning
• Authors: F. Llorente, L. Martino, J. Read, D. Delgado
• Abstract summary: This type of problem can be found in numerous real-world scenarios, including optimization and reinforcement learning. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A range of application scenarios is discussed, with special attention to the likelihood-free setting and reinforcement learning.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This survey gives an overview of Monte Carlo methodologies using surrogate models, for dealing with densities which are intractable, costly, and/or noisy. This type of problem can be found in numerous real-world scenarios, including stochastic optimization and reinforcement learning, where each evaluation of a density function may incur some computationally-expensive or even physical (real-world activity) cost, likely to give different results each time. The surrogate model does not incur this cost, but there are important trade-offs and considerations involved in the choice and design of such methodologies. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A modular scheme which encompasses the considered methods is also presented. A range of application scenarios is discussed, with special attention to the likelihood-free setting and reinforcement learning. Several numerical comparisons are also provided.

Related papers

• Scalable Structure Learning for Sparse Context-Specific Systems [0.0]
  We present an algorithm for learning context-specific models that scales to hundreds of variables. Our method is shown to perform well on synthetic data and real world examples.
  arXiv  Detail & Related papers  (2024-02-12T16:28:52Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• Impossibility Theorems for Feature Attribution [21.88229793890961]
  We show that for moderately rich model classes, any feature attribution method can provably fail to improve on random guessing for inferring model behaviour. Our results apply to common end-tasks such as characterizing local model behaviour, identifying spurious features, and algorithmic recourse.
  arXiv  Detail & Related papers  (2022-12-22T17:03:57Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Towards Better Understanding Attribution Methods [77.1487219861185]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
  arXiv  Detail & Related papers  (2022-05-20T20:50:17Z)
• Network Estimation by Mixing: Adaptivity and More [2.3478438171452014]
  We propose a mixing strategy that leverages available arbitrary models to improve their individual performances. The proposed method is computationally efficient and almost tuning-free. We show that the proposed method performs equally well as the oracle estimate when the true model is included as individual candidates.
  arXiv  Detail & Related papers  (2021-06-05T05:17:04Z)
• A generalized framework for active learning reliability: survey and benchmark [0.0]
  We propose a modular framework to build on-the-fly efficient active learning strategies. We devise 39 strategies for the solution of 20 reliability benchmark problems.
  arXiv  Detail & Related papers  (2021-06-03T09:33:59Z)
• Misclassification cost-sensitive ensemble learning: A unifying framework [7.90398448280017]
  Our contribution is a unifying framework that provides a comprehensive and insightful overview on cost-sensitive ensemble methods. Our framework contains natural extensions and generalisations of ideas across methods, be it AdaBoost, Bagging or Random Forest.
  arXiv  Detail & Related papers  (2020-07-14T21:18:33Z)
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology [66.06153115971732]
  We present a method for quantifying disentanglement that only uses the generative model. We empirically evaluate several state-of-the-art models across multiple datasets.
  arXiv  Detail & Related papers  (2020-06-05T20:54:11Z)
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review [66.37504201165159]
  This article provides a comprehensive study of the state-of-the-art of the topic. We highlight limitations, benefits, connections and differences among the different techniques. Problems and possible solutions with the use of improper priors are also described.
  arXiv  Detail & Related papers  (2020-05-17T18:31:58Z)
• There and Back Again: Revisiting Backpropagation Saliency Methods [87.40330595283969]
  Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample. A popular class of such methods is based on backpropagating a signal and analyzing the resulting gradient. We propose a single framework under which several such methods can be unified.
  arXiv  Detail & Related papers  (2020-04-06T17:58:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.