Fast kernel methods for Data Quality Monitoring as a goodness-of-fit test

• URL: http://arxiv.org/abs/2303.05413v1
• Date: Thu, 9 Mar 2023 16:59:35 GMT
• Title: Fast kernel methods for Data Quality Monitoring as a goodness-of-fit test
• Authors: Gaia Grosso, Nicol\`o Lai, Marco Letizia, Jacopo Pazzini, Marco Rando, Lorenzo Rosasco, Andrea Wulzer, Marco Zanetti
• Abstract summary: We propose a machine learning approach for monitoring particle detectors in real-time. The goal is to assess the compatibility of incoming experimental data with a reference dataset, characterising the data behaviour under normal circumstances. The model is based on a modern implementation of kernel methods, nonparametric algorithms that can learn any continuous function given enough data.
• Score: 10.882743697472755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We here propose a machine learning approach for monitoring particle detectors in real-time. The goal is to assess the compatibility of incoming experimental data with a reference dataset, characterising the data behaviour under normal circumstances, via a likelihood-ratio hypothesis test. The model is based on a modern implementation of kernel methods, nonparametric algorithms that can learn any continuous function given enough data. The resulting approach is efficient and agnostic to the type of anomaly that may be present in the data. Our study demonstrates the effectiveness of this strategy on multivariate data from drift tube chamber muon detectors.

Related papers

• Research on Dynamic Data Flow Anomaly Detection based on Machine Learning [11.526496773281938]
  In this study, the unsupervised learning method is employed to identify anomalies in dynamic data flows. By clustering similar data, the model is able to detect data behaviour that deviates significantly from normal traffic without the need for labelled data. Notably, it demonstrates robust and adaptable performance, particularly in the context of unbalanced data.
  arXiv  Detail & Related papers  (2024-09-23T08:19:15Z)
• Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution [62.71425232332837]
  We show that training amortized models with noisy labels is inexpensive and surprisingly effective. This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
  arXiv  Detail & Related papers  (2024-01-29T03:42:37Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Unsupervised Anomaly Detection via Nonlinear Manifold Learning [0.0]
  Anomalies are samples that significantly deviate from the rest of the data and their detection plays a major role in building machine learning models. We introduce a robust, efficient, and interpretable methodology based on nonlinear manifold learning to detect anomalies in unsupervised settings.
  arXiv  Detail & Related papers  (2023-06-15T18:48:10Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• A Data-Driven Method for Automated Data Superposition with Applications in Soft Matter Science [0.0]
  We develop a data-driven, non-parametric method for superposing experimental data with arbitrary coordinate transformations. Our method produces interpretable data-driven models that may inform applications such as materials classification, design, and discovery.
  arXiv  Detail & Related papers  (2022-04-20T14:58:04Z)
• Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems [2.4554686192257424]
  We propose a framework for learning continuous representations from a sample of multidimensional functional data. We show that the resulting estimation problem can be solved efficiently by the tensor decomposition. We conclude with a real data application in neuroimaging.
  arXiv  Detail & Related papers  (2021-07-30T16:02:15Z)
• Outlier detection in multivariate functional data through a contaminated mixture model [0.0]
  This work is motivated by an application in an industrial context, where the activity of sensors is recorded at a high frequency. The objective is to automatically detect abnormal measurement behaviour.
  arXiv  Detail & Related papers  (2021-06-14T08:17:42Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection [55.888835686183995]
  We propose a neural network-based meta-learning method for supervised anomaly detection. We experimentally demonstrate that the proposed method achieves better performance than existing anomaly detection and few-shot learning methods.
  arXiv  Detail & Related papers  (2021-03-01T01:43:04Z)
• Unsupervised machine learning of topological phase transitions from experimental data [52.77024349608834]
  We apply unsupervised machine learning techniques to experimental data from ultracold atoms. We obtain the topological phase diagram of the Haldane model in a completely unbiased fashion. Our work provides a benchmark for unsupervised detection of new exotic phases in complex many-body systems.
  arXiv  Detail & Related papers  (2021-01-14T16:38:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.