Secure Platform for Processing Sensitive Data on Shared HPC Systems
- URL: http://arxiv.org/abs/2103.14679v1
- Date: Fri, 26 Mar 2021 18:30:33 GMT
- Title: Secure Platform for Processing Sensitive Data on Shared HPC Systems
- Authors: Michel Scheerman, Narges Zarrabi, Martijn Kruiten, Maxime Mog\'e,
Lykle Voort, Annette Langedijk, Ruurd Schoonhoven, Tom Emery
- Abstract summary: High performance computing clusters pose challenges for processing sensitive data.
In this work we present a novel method for creating secure computing environments on traditional multi-tenant high-performance computing clusters.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: High performance computing clusters operating in shared and batch mode pose
challenges for processing sensitive data. In the meantime, the need for secure
processing of sensitive data on HPC system is growing. In this work we present
a novel method for creating secure computing environments on traditional
multi-tenant high-performance computing clusters. Our platform as a service
provides a customizable, virtualized solution using PCOCC and SLURM to meet
strict security requirements without modifying the exist-ing HPC
infrastructure. We show how this platform has been used in real-world research
applications from different research domains. The solution is scalable by
design with low performance overhead and can be generalized for processing
sensitive data on shared HPC systems imposing high security criteria
Related papers
- Hydra: Brokering Cloud and HPC Resources to Support the Execution of Heterogeneous Workloads at Scale [1.474723404975345]
Hydra is an intra cross-cloud HPC brokering system capable of concurrently acquiring resources from commercial private cloud and HPC platforms.
We present Hydra an intra cross-cloud HPC brokering system capable of concurrently acquiring resources from commercial private cloud and HPC platforms.
arXiv Detail & Related papers (2024-07-16T17:59:46Z) - Chat AI: A Seamless Slurm-Native Solution for HPC-Based Services [0.3124884279860061]
Large language models (LLMs) allow researchers to run open-source or custom fine-tuned LLMs and ensure users that their data remains private and is not stored without their consent.
We propose an implementation consisting of a web service that runs on a cloud VM with secure access to a scalable backend running a multitude of AI models on HPC systems.
In order to ensure the security of the HPC system, we use the SSH ForceCommand directive to construct a robust circuit breaker.
arXiv Detail & Related papers (2024-06-27T12:08:21Z) - Hook-in Privacy Techniques for gRPC-based Microservice Communication [0.0]
gRPC is at the heart of modern distributed system architectures.
Despite its widespread adoption, gRPC lacks any advanced privacy techniques beyond transport and basic token-based authentication.
We propose a novel approach for integrating such advanced privacy techniques into the gRPC framework in a practically viable way.
arXiv Detail & Related papers (2024-04-08T15:18:42Z) - Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS)
FLEKD enables a more flexible aggregation method than conventional model fusion techniques.
Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
arXiv Detail & Related papers (2024-01-22T14:16:37Z) - Unsupervised KPIs-Based Clustering of Jobs in HPC Data Centers [0.0]
Key Performance Indicators (KPIs) generate a huge number of monitoring tasks that give data about CPU usage, memory usage, network traffic, or other sensors that monitor hardware.
The main contribution in this paper is to identify which metric/s (KPIs) is/are the most appropriate to identify/classify different types of jobs according to their behavior in the HPC system.
We have concluded that (i. those metrics (KPIs) related to the Network (interface) traffic monitoring provide the best cohesion and separation to cluster HPC jobs, and (ii. hierarchical clustering algorithms are the most suitable for this task
arXiv Detail & Related papers (2023-12-11T17:31:46Z) - Scaling #DNN-Verification Tools with Efficient Bound Propagation and
Parallel Computing [57.49021927832259]
Deep Neural Networks (DNNs) are powerful tools that have shown extraordinary results in many scenarios.
However, their intricate designs and lack of transparency raise safety concerns when applied in real-world applications.
Formal Verification (FV) of DNNs has emerged as a valuable solution to provide provable guarantees on the safety aspect.
arXiv Detail & Related papers (2023-12-10T13:51:25Z) - One nine availability of a Photonic Quantum Computer on the Cloud toward
HPC integration [0.8961191069175432]
In November 2022, we introduced the first cloud-accessible general-purpose quantum computer based on single photons.
We describe the design and implementation of our cloud-accessible quantum computing platform, and demonstrate one nine availability (92 for external users during a six-month period, higher than most online services)
This work lay the foundation for advancing quantum computing accessibility and usability in hybrid HPC-QC infrastructures.
arXiv Detail & Related papers (2023-08-28T13:47:39Z) - SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines.
This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
arXiv Detail & Related papers (2021-12-22T14:45:37Z) - Towards AIOps in Edge Computing Environments [60.27785717687999]
This paper describes the system design of an AIOps platform which is applicable in heterogeneous, distributed environments.
It is feasible to collect metrics with a high frequency and simultaneously run specific anomaly detection algorithms directly on edge devices.
arXiv Detail & Related papers (2021-02-12T09:33:00Z) - Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field.
We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem.
We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
arXiv Detail & Related papers (2020-06-17T18:14:30Z) - A Privacy-Preserving Distributed Architecture for
Deep-Learning-as-a-Service [68.84245063902908]
This paper introduces a novel distributed architecture for deep-learning-as-a-service.
It is able to preserve the user sensitive data while providing Cloud-based machine and deep learning services.
arXiv Detail & Related papers (2020-03-30T15:12:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.