Optimized Three Deep Learning Models Based-PSO Hyperparameters for Beijing PM2.5 Prediction

• URL: http://arxiv.org/abs/2306.07296v1
• Date: Sat, 10 Jun 2023 16:06:44 GMT
• Title: Optimized Three Deep Learning Models Based-PSO Hyperparameters for Beijing PM2.5 Prediction
• Authors: Andri Pranolo, Yingchi Mao, Aji Prasetya Wibawa, Agung Bella Putra Utama, Felix Andika Dwiyanto
• Abstract summary: This research attempts to optimize the deep learning architecture of Long short term memory (LSTM), Convolutional neural network (CNN), and Multilayer perceptron (MLP) Beijing PM2.5 datasets was analyzed to measure the performance of the proposed models. A recommendation for air pollution management could be generated by using these optimized models.
• Score: 1.3649494534428745
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep learning is a machine learning approach that produces excellent performance in various applications, including natural language processing, image identification, and forecasting. Deep learning network performance depends on the hyperparameter settings. This research attempts to optimize the deep learning architecture of Long short term memory (LSTM), Convolutional neural network (CNN), and Multilayer perceptron (MLP) for forecasting tasks using Particle swarm optimization (PSO), a swarm intelligence-based metaheuristic optimization methodology: Proposed M-1 (PSO-LSTM), M-2 (PSO-CNN), and M-3 (PSO-MLP). Beijing PM2.5 datasets was analyzed to measure the performance of the proposed models. PM2.5 as a target variable was affected by dew point, pressure, temperature, cumulated wind speed, hours of snow, and hours of rain. The deep learning network inputs consist of three different scenarios: daily, weekly, and monthly. The results show that the proposed M-1 with three hidden layers produces the best results of RMSE and MAPE compared to the proposed M-2, M-3, and all the baselines. A recommendation for air pollution management could be generated by using these optimized models

Related papers

• Optimization-based Structural Pruning for Large Language Models without Back-Propagation [57.9629676017527]
  We propose an optimization-based structural pruning on Large-Language Models (LLMs) Our method learns the pruning masks in a probabilistic space directly by optimizing the loss of the pruned model. Our method operates for 2.7 hours with around 35GB memory for the 13B models on a single A100 GPU, and our pruned models outperform the state-of-the-arts w.r.t. perplexity.
  arXiv  Detail & Related papers  (2024-06-15T09:31:03Z)
• Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models [73.48675708831328]
  We propose a novel parameter and computation efficient tuning method for Multi-modal Large Language Models (MLLMs) The Efficient Attention Skipping (EAS) method evaluates the attention redundancy and skips the less important MHAs to speed up inference. The experiments show that EAS not only retains high performance and parameter efficiency, but also greatly speeds up inference speed.
  arXiv  Detail & Related papers  (2024-03-22T14:20:34Z)
• Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation [41.38091115195305]
  This work presents the experimental implementation of a deep neural network (DNN) based nonlinear MPC for Homogeneous Charge Compression Ignition (HCCI) combustion control. Using the acados software package to enable the real-time implementation of the MPC on an ARM Cortex A72, the optimization calculations are completed within 1.4 ms. The IMEP trajectory following of the developed controller was excellent, with a root-mean-square error of 0.133 bar, in addition to observing process constraints.
  arXiv  Detail & Related papers  (2023-10-12T15:03:50Z)
• Deep Learning for Day Forecasts from Sparse Observations [60.041805328514876]
  Deep neural networks offer an alternative paradigm for modeling weather conditions. MetNet-3 learns from both dense and sparse data sensors and makes predictions up to 24 hours ahead for precipitation, wind, temperature and dew point. MetNet-3 has a high temporal and spatial resolution, respectively, up to 2 minutes and 1 km as well as a low operational latency.
  arXiv  Detail & Related papers  (2023-06-06T07:07:54Z)
• Improving Urban Flood Prediction using LSTM-DeepLabv3+ and Bayesian Optimization with Spatiotemporal feature fusion [7.790241122137617]
  This study presented a CNN-RNN hybrid feature fusion modelling approach for urban flood prediction. It integrated the strengths of CNNs in processing spatial features and RNNs in analyzing different dimensions of time sequences.
  arXiv  Detail & Related papers  (2023-04-19T22:00:04Z)
• Development, Optimization, and Deployment of Thermal Forward Vision Systems for Advance Vehicular Applications on Edge Devices [0.3058685580689604]
  We have proposed a thermal tiny-YOLO multi-class object detection (TTYMOD) system as a smart forward sensing system using an end-to-end YOLO deep learning framework. The system is trained on large-scale thermal public as well as newly gathered novel open-sourced dataset comprising of more than 35,000 distinct thermal frames. The efficacy of a thermally tuned nano network is quantified using various qualitative metrics which include mean precision, frames per second rate, and average inference time.
  arXiv  Detail & Related papers  (2023-01-18T15:45:33Z)
• Multi-objective hyperparameter optimization with performance uncertainty [62.997667081978825]
  This paper presents results on multi-objective hyperparameter optimization with uncertainty on the evaluation of Machine Learning algorithms. We combine the sampling strategy of Tree-structured Parzen Estimators (TPE) with the metamodel obtained after training a Gaussian Process Regression (GPR) with heterogeneous noise. Experimental results on three analytical test functions and three ML problems show the improvement over multi-objective TPE and GPR.
  arXiv  Detail & Related papers  (2022-09-09T14:58:43Z)
• Bayesian Neural Network Language Modeling for Speech Recognition [59.681758762712754]
  State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex. In this paper, an overarching full Bayesian learning framework is proposed to account for the underlying uncertainty in LSTM-RNN and Transformer LMs.
  arXiv  Detail & Related papers  (2022-08-28T17:50:19Z)
• CPM-2: Large-scale Cost-effective Pre-trained Language Models [71.59893315671997]
  We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference. We introduce knowledge inheritance to accelerate the pre-training process by exploiting existing PLMs instead of training models from scratch. We implement a new inference toolkit, namely InfMoE, for using large-scale PLMs with limited computational resources.
  arXiv  Detail & Related papers  (2021-06-20T15:43:54Z)
• Particle Swarm Optimized Federated Learning For Industrial IoT and Smart City Services [9.693848515371268]
  We propose a Particle Swarm Optimization (PSO)-based technique to optimize the hyper parameter settings for the local Machine Learning models. We evaluate the performance of our proposed technique using two case studies.
  arXiv  Detail & Related papers  (2020-09-05T16:20:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.