DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments

• URL: http://arxiv.org/abs/2501.19237v1
• Date: Fri, 31 Jan 2025 15:51:41 GMT
• Title: DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments
• Authors: Arsenii Gavrikov, Julián García Pardiñas, Alberto Garfagnini,
• Abstract summary: We present novel, interpretable, robust, and scalable DQM algorithms designed to automate anomaly detection. Our approach constructs evolving histogram templates with built-in uncertainties, featuring a statistical variant. Experiments on synthetic datasets demonstrate the high accuracy, adaptability, and interpretability of these methods.
• Score: 0.0
• License:
• Abstract: Ensuring reliable data collection in large-scale particle physics experiments demands Data Quality Monitoring (DQM) procedures to detect possible detector malfunctions and preserve data integrity. Traditionally, this resource-intensive task has been handled by human shifters that struggle with frequent changes in operational conditions. We present novel, interpretable, robust, and scalable DQM algorithms designed to automate anomaly detection in time-dependent settings. Our approach constructs evolving histogram templates with built-in uncertainties, featuring both a statistical variant - extending the classical Exponentially Weighted Moving Average (EWMA) - and a machine learning (ML)-enhanced version that leverages a transformer encoder for improved adaptability. Experimental validations on synthetic datasets demonstrate the high accuracy, adaptability, and interpretability of these methods, with the statistical variant being commissioned in the LHCb experiment at the Large Hadron Collider, underscoring its real-world impact. The code used in this study is available at https://github.com/ArseniiGav/DINAMO.

Related papers

• Predictive Maintenance Model Based on Anomaly Detection in Induction Motors: A Machine Learning Approach Using Real-Time IoT Data [0.0]
  In this work, we demonstrate a novel anomaly detection system on induction motors used in pumps, compressors, fans, and other industrial machines. We use a combination of pre-processing techniques and machine learning (ML) models with a low computational cost.
  arXiv  Detail & Related papers  (2023-10-15T18:43:45Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)
• Evaluating the Transferability of Machine-Learned Force Fields for Material Property Modeling [2.494740426749958]
  We present a more comprehensive set of benchmarking tests for evaluating the transferability of machine-learned force fields. We use a graph neural network (GNN)-based force field coupled with the OpenMM package to carry out MD simulations for Argon. Our results show that the model can accurately capture the behavior of the solid phase only when the configurations from the solid phase are included in the training dataset.
  arXiv  Detail & Related papers  (2023-01-10T00:25:48Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility [1.014927488137914]
  Probabilistic generative models are attractive for scientific modeling because their inferred parameters can be used to generate hypotheses and design experiments. Supervised Variational Autoencoders (SVAEs) have previously been used for this purpose. We develop a second order supervision framework (SOS-VAE) that influences the decoder to induce a predictive latent representation.
  arXiv  Detail & Related papers  (2021-09-09T20:55:38Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Quantum-tailored machine-learning characterization of a superconducting qubit [50.591267188664666]
  We develop an approach to characterize the dynamics of a quantum device and learn device parameters. This approach outperforms physics-agnostic recurrent neural networks trained on numerically generated and experimental data. This demonstration shows how leveraging domain knowledge improves the accuracy and efficiency of this characterization task.
  arXiv  Detail & Related papers  (2021-06-24T15:58:57Z)
• Self-awareness in intelligent vehicles: Feature based dynamic Bayesian models for abnormality detection [4.251384905163326]
  This paper aims to introduce a novel method to develop self-awareness in autonomous vehicles. Time-series data from the vehicles are used to develop the data-driven Dynamic Bayesian Network (DBN) models. An initial level collective awareness model that can perform joint anomaly detection in co-operative tasks is proposed.
  arXiv  Detail & Related papers  (2020-10-29T09:29:47Z)
• Anomaly Detection With Conditional Variational Autoencoders [1.3541554606406663]
  We exploit the deep conditional variational autoencoder (CVAE) and we define an original loss function together with a metric that targets hierarchically structured data. Our motivating application is a real world problem: monitoring the trigger system which is a basic component of many particle physics experiments at the CERN Large Hadron Collider.
  arXiv  Detail & Related papers  (2020-10-12T08:39:37Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.