Evaluating generative models in high energy physics

• URL: http://arxiv.org/abs/2211.10295v2
• Date: Fri, 21 Apr 2023 16:14:36 GMT
• Title: Evaluating generative models in high energy physics
• Authors: Raghav Kansal and Anni Li and Javier Duarte and Nadezda Chernyavskaya and Maurizio Pierini and Breno Orzari and Thiago Tomei
• Abstract summary: We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models. We propose two new metrics, the Fr'echet and kernel physics distances (FPD and KPD, respectively), and perform a variety of experiments measuring their performance. We demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model.
• Score: 7.545095780512178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fr\'echet and kernel physics distances (FPD and KPD, respectively), and perform a variety of experiments measuring their performance on simple Gaussian-distributed, and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source JetNet Python library.

Related papers

• Refereeing the Referees: Evaluating Two-Sample Tests for Validating Generators in Precision Sciences [0.0]
  One-dimensional tests provide a level of sensitivity comparable to other multivariate metrics, but with significantly lower computational cost. This methodology offers an efficient, standardized tool for model comparison and can serve as a benchmark for more advanced tests.
  arXiv  Detail & Related papers  (2024-09-24T13:58:46Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• Multivariate Physics-Informed Convolutional Autoencoder for Anomaly Detection in Power Distribution Systems with High Penetration of DERs [0.0]
  This paper proposes a physics-informed convolutional autoencoder (PIConvAE) model to detect cyber anomalies in power distribution systems with unbalanced configurations and high penetration of DERs. The performance of the proposed model is evaluated on two unbalanced power distribution grids, IEEE 123-bus system and a real-world feeder in Riverside, CA.
  arXiv  Detail & Related papers  (2024-06-05T04:28:57Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Deep Generative Models for Ultra-High Granularity Particle Physics Detector Simulation: A Voyage From Emulation to Extrapolation [0.0]
  This thesis aims to overcome this challenge for the Pixel Vertex Detector (PXD) at the Belle II experiment. This study introduces, for the first time, the results of using deep generative models for ultra-high granularity detector simulation in Particle Physics.
  arXiv  Detail & Related papers  (2024-03-05T23:12:47Z)
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review [0.0]
  Signatures from particle physics detectors are low-level objects (such as energy depositions or tracks) encoding the physics of collisions. The complete simulation of them in a detector is a computational and storage-intensive task. We conduct a comprehensive and exhaustive taxonomic review of the existing literature on the simulation of detector signatures.
  arXiv  Detail & Related papers  (2023-12-15T08:27:39Z)
• Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [64.95663299945171]
  Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming. There exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. We propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs.
  arXiv  Detail & Related papers  (2023-09-10T22:05:24Z)
• MATNet: Multi-Level Fusion Transformer-Based Model for Day-Ahead PV Generation Forecasting [0.47518865271427785]
  MATNet is a novel self-attention transformer-based architecture for PV power generation forecasting. It consists of a hybrid approach that combines the AI paradigm with the prior physical knowledge of PV power generation. Results show that our proposed architecture significantly outperforms the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-17T14:03:09Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.