Integrating Deep Learning in Domain Sciences at Exascale
- URL: http://arxiv.org/abs/2011.11188v1
- Date: Mon, 23 Nov 2020 03:09:58 GMT
- Title: Integrating Deep Learning in Domain Sciences at Exascale
- Authors: Rick Archibald, Edmond Chow, Eduardo D'Azevedo, Jack Dongarra, Markus
Eisenbach, Rocco Febbo, Florent Lopez, Daniel Nichols, Stanimire Tomov, Kwai
Wong, and Junqi Yin
- Abstract summary: We evaluate existing packages for their ability to run deep learning models and applications on large-scale HPC systems efficiently.
We propose new asynchronous parallelization and optimization techniques for current large-scale heterogeneous systems.
We present illustrations and potential solutions for enhancing traditional compute- and data-intensive applications with AI.
- Score: 2.241545093375334
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper presents some of the current challenges in designing deep learning
artificial intelligence (AI) and integrating it with traditional
high-performance computing (HPC) simulations. We evaluate existing packages for
their ability to run deep learning models and applications on large-scale HPC
systems efficiently, identify challenges, and propose new asynchronous
parallelization and optimization techniques for current large-scale
heterogeneous systems and upcoming exascale systems. These developments, along
with existing HPC AI software capabilities, have been integrated into MagmaDNN,
an open-source HPC deep learning framework. Many deep learning frameworks are
targeted at data scientists and fall short in providing quality integration
into existing HPC workflows. This paper discusses the necessities of an HPC
deep learning framework and how those needs can be provided (e.g., as in
MagmaDNN) through a deep integration with existing HPC libraries, such as MAGMA
and its modular memory management, MPI, CuBLAS, CuDNN, MKL, and HIP.
Advancements are also illustrated through the use of algorithmic enhancements
in reduced- and mixed-precision, as well as asynchronous optimization methods.
Finally, we present illustrations and potential solutions for enhancing
traditional compute- and data-intensive applications at ORNL and UTK with AI.
The approaches and future challenges are illustrated in materials science,
imaging, and climate applications.
Related papers
- Quantum Computing Enhanced Service Ecosystem for Simulation in
Manufacturing [57.497038270123774]
We propose a framework for a quantum computing-enhanced service ecosystem for simulation in manufacturing.
We analyse two high-value use cases with the aim of a quantitative evaluation of these new computing paradigms for industrially-relevant settings.
arXiv Detail & Related papers (2024-01-19T11:04:14Z) - Machine Learning Insides OptVerse AI Solver: Design Principles and
Applications [74.67495900436728]
We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver.
We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem.
We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
arXiv Detail & Related papers (2024-01-11T15:02:15Z) - Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement
Learning Approach [58.911515417156174]
We propose a new definition of Age of Information (AoI) and, based on the redefined AoI, we formulate an online AoI problem for MEC systems.
We introduce Post-Decision States (PDSs) to exploit the partial knowledge of the system's dynamics.
We also combine PDSs with deep RL to further improve the algorithm's applicability, scalability, and robustness.
arXiv Detail & Related papers (2023-12-01T01:30:49Z) - CSM-H-R: A Context Modeling Framework in Supporting Reasoning Automation for Interoperable Intelligent Systems and Privacy Protection [0.07499722271664144]
We propose a novel framework for automation of High Level Context (HLC) reasoning across intelligent systems at scale.
The design of the framework supports the sharing and inter context among intelligent systems and the components for handling CSMs and the management of hierarchy, relationship, and transition.
The implementation of the framework experiments on the HLC reasoning into vector and matrix computing and presents the potential to reach next level of automation.
arXiv Detail & Related papers (2023-08-21T22:21:15Z) - Hyperparameter optimization, quantum-assisted model performance
prediction, and benchmarking of AI-based High Energy Physics workloads using
HPC [0.0]
This work studies the potential of using model performance prediction to aid the HPO process carried out on High Performance Computing systems.
A quantum annealer is used to train the performance predictor and a method is proposed to overcome some of the problems derived from the current limitations in quantum systems.
Results are presented from the development of a containerized benchmark based on an AI-model for collision event reconstruction.
arXiv Detail & Related papers (2023-03-27T09:55:33Z) - Deep learning applied to computational mechanics: A comprehensive
review, state of the art, and the classics [77.34726150561087]
Recent developments in artificial neural networks, particularly deep learning (DL), are reviewed in detail.
Both hybrid and pure machine learning (ML) methods are discussed.
History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics.
arXiv Detail & Related papers (2022-12-18T02:03:00Z) - SAIH: A Scalable Evaluation Methodology for Understanding AI Performance
Trend on HPC Systems [18.699431277588637]
We propose a scalable evaluation methodology (SAIH) for analyzing the AI performance trend of HPC systems.
As the data and model constantly scale, we can investigate the trend and range of AI performance on HPC systems.
arXiv Detail & Related papers (2022-12-07T02:42:29Z) - Towards a Dynamic Composability Approach for using Heterogeneous Systems
in Remote Sensing [0.0]
We present a novel approach for using composable systems in the intersection between scientific computing, artificial intelligence (AI), and remote sensing domain.
We describe the architecture of a first working example of a composable infrastructure that federates Expanse, an NSF-funded supercomputer, with Nautilus, a geo-distributed cluster.
arXiv Detail & Related papers (2022-11-13T14:48:00Z) - AI-coupled HPC Workflows [1.5469452301122175]
Introduction to AI/ML models into the traditional HPC has been an enabler of highly accurate modeling.
Various modes of integrating AI/ML models to HPC computations, resulting in diverse types of AI-coupled HPC.
arXiv Detail & Related papers (2022-08-24T19:16:43Z) - Reconfigurable Intelligent Surface Assisted Mobile Edge Computing with
Heterogeneous Learning Tasks [53.1636151439562]
Mobile edge computing (MEC) provides a natural platform for AI applications.
We present an infrastructure to perform machine learning tasks at an MEC with the assistance of a reconfigurable intelligent surface (RIS)
Specifically, we minimize the learning error of all participating users by jointly optimizing transmit power of mobile users, beamforming vectors of the base station, and the phase-shift matrix of the RIS.
arXiv Detail & Related papers (2020-12-25T07:08:50Z) - Deep Multi-Task Learning for Cooperative NOMA: System Design and
Principles [52.79089414630366]
We develop a novel deep cooperative NOMA scheme, drawing upon the recent advances in deep learning (DL)
We develop a novel hybrid-cascaded deep neural network (DNN) architecture such that the entire system can be optimized in a holistic manner.
arXiv Detail & Related papers (2020-07-27T12:38:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.