Goodness of fit by Neyman-Pearson testing

• URL: http://arxiv.org/abs/2305.14137v2
• Date: Fri, 15 Mar 2024 16:30:28 GMT
• Title: Goodness of fit by Neyman-Pearson testing
• Authors: Gaia Grosso, Marco Letizia, Maurizio Pierini, Andrea Wulzer,
• Abstract summary: Neyman-Pearson strategy for hypothesis testing can be employed for goodness of fit if the alternative hypothesis is selected from data. New Physics Learning Machine (NPLM) methodology has been developed to target the detection of new physical effects in the context of high energy physics collider experiments. NPLM emerges as the more sensitive test to small departures of the data from the expected distribution and not biased towards detecting specific types of anomalies.
• Score: 1.000352363234953
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Neyman-Pearson strategy for hypothesis testing can be employed for goodness of fit if the alternative hypothesis is selected from data by exploring a rich parametrised family of models, while controlling the impact of statistical fluctuations. The New Physics Learning Machine (NPLM) methodology has been developed as a concrete implementation of this idea, to target the detection of new physical effects in the context of high energy physics collider experiments. In this paper we conduct a comparison of this approach to goodness of fit with others, in particular with classifier-based strategies that share strong similarities with NPLM. From our comparison, NPLM emerges as the more sensitive test to small departures of the data from the expected distribution and not biased towards detecting specific types of anomalies. These features make it suited for agnostic searches for new physics at collider experiments. Its deployment in other scientific and industrial scenarios should be investigated.

Related papers

• Multiple testing for signal-agnostic searches of new physics with machine learning [0.0]
  We consider the question of how to enhance signal-agnostic searches by leveraging multiple testing strategies. We focus on the New Physics Learning Machine, a methodology to perform a signal-agnostic likelihood-ratio test.
  arXiv  Detail & Related papers  (2024-08-22T11:14:37Z)
• Uncertainty Measurement of Deep Learning System based on the Convex Hull of Training Sets [0.13265175299265505]
  We propose To-hull Uncertainty and Closure Ratio, which measures an uncertainty of trained model based on the convex hull of training data. It can observe the positional relation between the convex hull of the learned data and an unseen sample and infer how extrapolate the sample is from the convex hull.
  arXiv  Detail & Related papers  (2024-05-25T06:25:24Z)
• Online simulator-based experimental design for cognitive model selection [74.76661199843284]
  We propose BOSMOS: an approach to experimental design that can select between computational models without tractable likelihoods. In simulated experiments, we demonstrate that the proposed BOSMOS technique can accurately select models in up to 2 orders of magnitude less time than existing LFI alternatives.
  arXiv  Detail & Related papers  (2023-03-03T21:41:01Z)
• Classical models may be a better explanation of the Jiuzhang 1.0 Gaussian Boson Sampler than its targeted squeezed light model [0.0]
  We propose an alternative classical hypothesis for the validation of the Jiuzhang 1.0 and Jiuzhang 2.0 experiments. Our results provide a new hypothesis that should be considered in the validation of future GBS experiments.
  arXiv  Detail & Related papers  (2022-07-20T17:39:44Z)
• Conformal prediction for the design problem [72.14982816083297]
  In many real-world deployments of machine learning, we use a prediction algorithm to choose what data to test next. In such settings, there is a distinct type of distribution shift between the training and test data. We introduce a method to quantify predictive uncertainty in such settings.
  arXiv  Detail & Related papers  (2022-02-08T02:59:12Z)
• Variance Minimization in the Wasserstein Space for Invariant Causal Prediction [72.13445677280792]
  In this work, we show that the approach taken in ICP may be reformulated as a series of nonparametric tests that scales linearly in the number of predictors. Each of these tests relies on the minimization of a novel loss function that is derived from tools in optimal transport theory. We prove under mild assumptions that our method is able to recover the set of identifiable direct causes, and we demonstrate in our experiments that it is competitive with other benchmark causal discovery algorithms.
  arXiv  Detail & Related papers  (2021-10-13T22:30:47Z)
• Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
  Inferring the parameters of a model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations. We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
  arXiv  Detail & Related papers  (2021-02-12T12:23:13Z)
• With Little Power Comes Great Responsibility [54.96675741328462]
  Underpowered experiments make it more difficult to discern the difference between statistical noise and meaningful model improvements. Small test sets mean that most attempted comparisons to state of the art models will not be adequately powered. For machine translation, we find that typical test sets of 2000 sentences have approximately 75% power to detect differences of 1 BLEU point.
  arXiv  Detail & Related papers  (2020-10-13T18:00:02Z)
• Balance-Subsampled Stable Prediction [55.13512328954456]
  We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
  arXiv  Detail & Related papers  (2020-06-08T07:01:38Z)
• A quantum algorithm for model independent searches for new physics [0.0]
  We propose a novel quantum technique to search for unmodelled anomalies in multi-dimensional binned collider data. We associate an Ising lattice spin site with each bin, with the Ising Hamiltonian suitably constructed from the observed data. In order to capture spatially correlated anomalies in the data, we introduce spin-spin interactions between neighboring sites.
  arXiv  Detail & Related papers  (2020-03-04T16:51:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.