MEDIC: a network for monitoring data quality in collider experiments

• URL: http://arxiv.org/abs/2511.18172v1
• Date: Sat, 22 Nov 2025 19:53:24 GMT
• Title: MEDIC: a network for monitoring data quality in collider experiments
• Authors: Juvenal Bassa, Arghya Chattopadhyay, Sudhir Malik, Mario Escabi Rivera,
• Abstract summary: Data Quality Monitoring (DQM) is a crucial component of particle physics experiments.<n>In this work, a simulation-driven approach to DQM is proposed, enabling the study and development of data-quality methodologies.<n>We introduce MEDIC, a neural network designed to learn detector behavior and perform DQM tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data Quality Monitoring (DQM) is a crucial component of particle physics experiments and ensures that the recorded data is of the highest quality, and suitable for subsequent physics analysis. Due to the extreme environmental conditions, unprecedented data volumes, and the sheer scale and complexity of the detectors, DQM orchestration has become a very challenging task. Therefore, the use of Machine Learning (ML) to automate anomaly detection, improve efficiency, and reduce human error in the process of collecting high-quality data is unavoidable. Since DQM relies on real experimental data, it is inherently tied to the specific detector substructure and technology in operation. In this work, a simulation-driven approach to DQM is proposed, enabling the study and development of data-quality methodologies in a controlled environment. Using a modified version of Delphes -- a fast, multi-purpose detector simulation -- the preliminary realization of a framework is demonstrated which leverages ML to identify detector anomalies as well as localize the malfunctioning components responsible. We introduce MEDIC (Monitoring for Event Data Integrity and Consistency), a neural network designed to learn detector behavior and perform DQM tasks to look for potential faults. Although the present implementation adopts a simplified setup for computational ease, where large detector regions are deliberately deactivated to mimic faults, this work represents an initial step toward a comprehensive ML-based DQM framework. The encouraging results underline the potential of simulation-driven studies as a foundation for developing more advanced, data-driven DQM systems for future particle detectors.

Related papers

• DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments [0.0]
  We present DINAMO: a novel, interpretable, robust, and scalable DQM framework.<n>Our approach constructs evolving histogram templates with built-in uncertainties.<n>The statistical variant is being commissioned in the LHCb experiment at the Large Hadron Collider.
  arXiv  Detail & Related papers  (2025-01-31T15:51:41Z)
• Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection [0.7767589715518638]
  Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task.<n>We present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks.<n>This research investigates the transferability of models trained on different sections of the Calorimeter of the Compact Muon Solenoid experiment at CERN.
  arXiv  Detail & Related papers  (2024-08-29T15:19:06Z)
• Human-in-the-loop Reinforcement Learning for Data Quality Monitoring in Particle Physics Experiments [0.0]
  We propose a proof-of-concept for applying human-in-the-loop Reinforcement Learning to automate the Data Quality Monitoring process. We show that random, unbiased noise in human classification can be reduced, leading to an improved accuracy over the baseline.
  arXiv  Detail & Related papers  (2024-05-24T12:52:46Z)
• Signal Detection in MIMO Systems with Hardware Imperfections: Message Passing on Neural Networks [101.59367762974371]
  In this paper, we investigate signal detection in multiple-input-multiple-output (MIMO) communication systems with hardware impairments. It is difficult to train a deep neural network (DNN) with limited pilot signals, hindering its practical applications. We design an efficient message passing based Bayesian signal detector, leveraging the unitary approximate message passing (UAMP) algorithm.
  arXiv  Detail & Related papers  (2022-10-08T04:32:58Z)
• Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
  We study robust event classification using imperfect real-world phasor measurement unit (PMU) data. We develop a novel machine learning framework for training robust event classifiers.
  arXiv  Detail & Related papers  (2021-10-19T17:41:43Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Real-World Anomaly Detection by using Digital Twin Systems and Weakly-Supervised Learning [3.0100975935933567]
  We present novel weakly-supervised approaches to anomaly detection for industrial settings. The approaches make use of a Digital Twin to generate a training dataset which simulates the normal operation of the machinery. The performance of the proposed methods is compared against various state-of-the-art anomaly detection algorithms on an application to a real-world dataset.
  arXiv  Detail & Related papers  (2020-11-12T10:15:56Z)
• Contextual-Bandit Anomaly Detection for IoT Data in Distributed Hierarchical Edge Computing [65.78881372074983]
  IoT devices can hardly afford complex deep neural networks (DNN) models, and offloading anomaly detection tasks to the cloud incurs long delay. We propose and build a demo for an adaptive anomaly detection approach for distributed hierarchical edge computing (HEC) systems. We show that our proposed approach significantly reduces detection delay without sacrificing accuracy, as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-04-15T06:13:33Z)
• Data-Driven Failure Prediction in Brittle Materials: A Phase-Field Based Machine Learning Framework [1.3858051019755282]
  Failure in brittle materials led by micro- to macro-cracks under repetitive or increasing loads is often catastrophic. We develop a supervised machine learning (ML) framework to predict failure in an isothermal, linear elastic and isotropic phase-field model. Results indicate that the proposed framework is capable of predicting failure with acceptable accuracy even in the presence of high noise levels.
  arXiv  Detail & Related papers  (2020-03-24T17:13:08Z)
• Adaptive Anomaly Detection for IoT Data in Hierarchical Edge Computing [71.86955275376604]
  We propose an adaptive anomaly detection approach for hierarchical edge computing (HEC) systems to solve this problem. We design an adaptive scheme to select one of the models based on the contextual information extracted from input data, to perform anomaly detection. We evaluate our proposed approach using a real IoT dataset, and demonstrate that it reduces detection delay by 84% while maintaining almost the same accuracy as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-01-10T05:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.