Accelerating Domain-aware Deep Learning Models with Distributed Training

• URL: http://arxiv.org/abs/2301.11787v1
• Date: Wed, 25 Jan 2023 22:59:47 GMT
• Title: Accelerating Domain-aware Deep Learning Models with Distributed Training
• Authors: Aishwarya Sarkar, Chaoqun Lu and Ali Jannesari
• Abstract summary: We present a novel distributed domain-aware network that utilizes domain-specific knowledge with improved model performance. From our analysis, the network effectively predicts high peaks in discharge measurements at watershed outlets with up to 4.1x speedup. Our approach achieved a 12.6x overall speedup and the mean prediction performance by 16%.
• Score: 0.8164433158925593
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in data-generating techniques led to an explosive growth of geo-spatiotemporal data. In domains such as hydrology, ecology, and transportation, interpreting the complex underlying patterns of spatiotemporal interactions with the help of deep learning techniques hence becomes the need of the hour. However, applying deep learning techniques without domain-specific knowledge tends to provide sub-optimal prediction performance. Secondly, training such models on large-scale data requires extensive computational resources. To eliminate these challenges, we present a novel distributed domain-aware spatiotemporal network that utilizes domain-specific knowledge with improved model performance. Our network consists of a pixel-contribution block, a distributed multiheaded multichannel convolutional (CNN) spatial block, and a recurrent temporal block. We choose flood prediction in hydrology as a use case to test our proposed method. From our analysis, the network effectively predicts high peaks in discharge measurements at watershed outlets with up to 4.1x speedup and increased prediction performance of up to 93\%. Our approach achieved a 12.6x overall speedup and increased the mean prediction performance by 16\%. We perform extensive experiments on a dataset of 23 watersheds in a northern state of the U.S. and present our findings.

Related papers

• Application of Long-Short Term Memory and Convolutional Neural Networks for Real-Time Bridge Scour Prediction [0.0]
  We exploit the power of deep learning algorithms to forecast scour depth variations around bridge piers based on historical sensor monitoring data. We investigated the performance of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) models for real-time scour forecasting.
  arXiv  Detail & Related papers  (2024-04-25T12:04:36Z)
• Rapid Flood Inundation Forecast Using Fourier Neural Operator [77.30160833875513]
  Flood inundation forecast provides critical information for emergency planning before and during flood events. High-resolution hydrodynamic modeling has become more accessible in recent years, however, predicting flood extents at the street and building levels in real-time is still computationally demanding. We present a hybrid process-based and data-driven machine learning (ML) approach for flood extent and inundation depth prediction.
  arXiv  Detail & Related papers  (2023-07-29T22:49:50Z)
• An evaluation of deep learning models for predicting water depth evolution in urban floods [59.31940764426359]
  We compare different deep learning models for prediction of water depth at high spatial resolution. Deep learning models are trained to reproduce the data simulated by the CADDIES cellular-automata flood model. Our results show that the deep learning models present in general lower errors compared to the other methods.
  arXiv  Detail & Related papers  (2023-02-20T16:08:54Z)
• Transfer learning to improve streamflow forecasts in data sparse regions [0.0]
  We study the methodology behind Transfer Learning (TL) through fine-tuning and parameter transferring for better generalization performance of streamflow prediction in data-sparse regions. We propose a standard recurrent neural network in the form of Long Short-Term Memory (LSTM) to fit on a sufficiently large source domain dataset. We present a methodology to implement transfer learning approaches for hydrologic applications by separating the spatial and temporal components of the model and training the model to generalize.
  arXiv  Detail & Related papers  (2021-12-06T14:52:53Z)
• Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations [86.5302150777089]
  Generative Adversarial Imputation Nets (GANs) and GAN-based techniques have attracted attention as unsupervised machine learning methods. We name our proposed method as Con Conval Generative Adversarial Imputation Nets (Conv-GAIN)
  arXiv  Detail & Related papers  (2021-11-03T03:50:48Z)
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining [58.10436813430554]
  Mini-batch training of graph neural networks (GNNs) requires a lot of computation and data movement. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised.
  arXiv  Detail & Related papers  (2021-10-16T02:41:35Z)
• Transfer Learning Approaches for Knowledge Discovery in Grid-based Geo-Spatiotemporal Data [1.2693545159861856]
  Extracting and analyzing geo-spatiotemporal features is crucial to recognize underlying causes of natural, events such as floods. We propose HydroDeep, an effectively reusable pretrained model to address this problem of transferring knowledge from one region to another.
  arXiv  Detail & Related papers  (2021-10-02T16:55:34Z)
• Estimating permeability of 3D micro-CT images by physics-informed CNNs based on DNS [1.6274397329511197]
  This paper presents a novel methodology for permeability prediction from micro-CT scans of geological rock samples. The training data set for CNNs dedicated to permeability prediction consists of permeability labels that are typically generated by classical lattice Boltzmann methods (LBM) We instead perform direct numerical simulation (DNS) by solving the stationary Stokes equation in an efficient and distributed-parallel manner.
  arXiv  Detail & Related papers  (2021-09-04T08:43:19Z)
• A Graph Convolutional Network with Signal Phasing Information for Arterial Traffic Prediction [63.470149585093665]
  arterial traffic prediction plays a crucial role in the development of modern intelligent transportation systems. Many existing studies on arterial traffic prediction only consider temporal measurements of flow and occupancy from loop sensors and neglect the rich spatial relationships between upstream and downstream detectors. We fill this gap by enhancing a deep learning approach, Diffusion Convolutional Recurrent Neural Network, with spatial information generated from signal timing plans at targeted intersections.
  arXiv  Detail & Related papers  (2020-12-25T01:40:29Z)
• Data-driven Flood Emulation: Speeding up Urban Flood Predictions by Deep Convolutional Neural Networks [0.0]
  This paper proposes that the prediction of maximum water depths can be considered as an image-to-image translation problem. The results are generated from input elevations using the information learned from data rather than by conducting simulations. The proposed neural network can potentially be applied to different but relevant problems including flood predictions for urban layout planning.
  arXiv  Detail & Related papers  (2020-04-17T16:44:46Z)
• Large Batch Training Does Not Need Warmup [111.07680619360528]
  Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications. In this paper, we propose a novel Complete Layer-wise Adaptive Rate Scaling (CLARS) algorithm for large-batch training. Based on our analysis, we bridge the gap and illustrate the theoretical insights for three popular large-batch training techniques.
  arXiv  Detail & Related papers  (2020-02-04T23:03:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.