Innovations in trigger and data acquisition systems for next-generation physics facilities

• URL: http://arxiv.org/abs/2203.07620v1
• Date: Tue, 15 Mar 2022 03:13:32 GMT
• Title: Innovations in trigger and data acquisition systems for next-generation physics facilities
• Authors: Rainer Bartoldus, Catrin Bernius, David W. Miller
• Abstract summary: Data-intensive physics facilities are increasingly reliant on heterogeneous and large-scale data processing systems. This White Paper aims to highlight the challenges that these facilities face in the design of the trigger and data acquisition instrumentation and systems.
• Score: 0.6445605125467573
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-intensive physics facilities are increasingly reliant on heterogeneous and large-scale data processing and computational systems in order to collect, distribute, process, filter, and analyze the ever increasing huge volumes of data being collected. Moreover, these tasks are often performed in hard real-time or quasi real-time processing pipelines that place extreme constraints on various parameters and design choices for those systems. Consequently, a large number and variety of challenges are faced to design, construct, and operate such facilities. This is especially true at the energy and intensity frontiers of particle physics where bandwidths of raw data can exceed 100 Tb/s of heterogeneous, high-dimensional data sourced from >300M individual sensors. Data filtering and compression algorithms deployed at these facilities often operate at the level of 1 part in $10^5$, and once executed, these algorithms drive the data curation process, further highlighting the critical roles that these systems have in the physics impact of those endeavors. This White Paper aims to highlight the challenges that these facilities face in the design of the trigger and data acquisition instrumentation and systems, as well as in their installation, commissioning, integration and operation, and in building the domain knowledge and technical expertise required to do so.

Related papers

• Earth System Data Cubes: Avenues for advancing Earth system research [4.408949931570938]
  Earth System Data Cubes ( ESDCs) have emerged as one suitable solution for transforming this flood of data into a simple yet robust format. ESDCs achieve this by organising data into an analysis-ready format with atemporal grid. There exist barriers to realising the full potential of data in light of novel cloud-based technologies.
  arXiv  Detail & Related papers  (2024-08-05T09:50:16Z)
• Enabling High Data Throughput Reinforcement Learning on GPUs: A Domain Agnostic Framework for Data-Driven Scientific Research [90.91438597133211]
  We introduce WarpSci, a framework designed to overcome crucial system bottlenecks in the application of reinforcement learning. We eliminate the need for data transfer between the CPU and GPU, enabling the concurrent execution of thousands of simulations.
  arXiv  Detail & Related papers  (2024-08-01T21:38:09Z)
• Data Augmentations in Deep Weight Spaces [89.45272760013928]
  We introduce a novel augmentation scheme based on the Mixup method. We evaluate the performance of these techniques on existing benchmarks as well as new benchmarks we generate.
  arXiv  Detail & Related papers  (2023-11-15T10:43:13Z)
• A spectrum of physics-informed Gaussian processes for regression in engineering [0.0]
  Despite the growing availability of sensing and data in general, we remain unable to fully characterise many in-service engineering systems and structures from a purely data-driven approach. This paper pursues the combination of machine learning technology and physics-based reasoning to enhance our ability to make predictive models with limited data.
  arXiv  Detail & Related papers  (2023-09-19T14:39:03Z)
• Linking the Dynamic PicoProbe Analytical Electron-Optical Beam Line / Microscope to Supercomputers [39.52789559084336]
  Dynamic PicoProbe at Argonne National Laboratory is undergoing upgrades that will enable it to produce up to 100s of GB of data per day. While this data is highly important for both fundamental science and industrial applications, there is currently limited on-site infrastructure to handle these high-volume data streams. We address this problem by providing a software architecture capable of supporting neighboring large-scale data transfers to neighboring supercomputers at the Argonne Leadership Computing Facility. This infrastructure supports expected workloads and also provides domain scientists the ability to reinterrogate data from past experiments to yield additional scientific value and derive new insights.
  arXiv  Detail & Related papers  (2023-08-25T23:07:58Z)
• LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting [65.71129509623587]
  Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning. However, the promising results achieved on current public datasets may not be applicable to practical scenarios. We introduce the LargeST benchmark dataset, which includes a total of 8,600 sensors in California with a 5-year time coverage.
  arXiv  Detail & Related papers  (2023-06-14T05:48:36Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• Towards an Interpretable Data-driven Trigger System for High-throughput Physics Facilities [7.939382824995354]
  We introduce a new data-driven approach for designing high- throughput data filtering and trigger systems. Our goal is to design a data-driven filtering system with a minimal run-time cost for determining which data event to keep. We introduce key insights from interpretable predictive modeling and cost-sensitive learning in order to account for non-local inefficiencies in the current paradigm.
  arXiv  Detail & Related papers  (2021-04-14T05:01:32Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review [68.15472610671748]
  Cloud computing has become a powerful and indispensable technology for complex, high performance and scalable computation. The rapid rate and volume of data creation has begun to pose significant challenges for data management and security. The design and deployment of intrusion detection systems (IDS) in the big data setting has, therefore, become a topic of importance.
  arXiv  Detail & Related papers  (2020-05-23T20:57:12Z)
• Towards an Integrated Platform for Big Data Analysis [4.5257812998381315]
  This paper presents the vision of an integrated plat-form for big data analysis that combines all these aspects. Main benefits of this approach are an enhanced scalability of the whole platform, a better parameterization of algorithms, and an improved usability during the end-to-end data analysis process.
  arXiv  Detail & Related papers  (2020-04-27T03:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.