Neural Network Compression for Reinforcement Learning Tasks

• URL: http://arxiv.org/abs/2405.07748v1
• Date: Mon, 13 May 2024 13:46:02 GMT
• Title: Neural Network Compression for Reinforcement Learning Tasks
• Authors: Dmitry A. Ivanov, Denis A. Larionov, Oleg V. Maslennikov, Vladimir V. Voevodin,
• Abstract summary: In real applications of Reinforcement Learning (RL), such as robotics, low latency and energy efficient inference is very desired. The use of sparsity and pruning for optimizing Neural Network inference, and particularly to improve energy and latency efficiency, is a standard technique. In this work, we perform a systematic investigation of applying these optimization techniques for different RL algorithms in different RL environments, yielding up to a 400-fold reduction in the size of neural networks.
• Score: 1.0124625066746595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In real applications of Reinforcement Learning (RL), such as robotics, low latency and energy efficient inference is very desired. The use of sparsity and pruning for optimizing Neural Network inference, and particularly to improve energy and latency efficiency, is a standard technique. In this work, we perform a systematic investigation of applying these optimization techniques for different RL algorithms in different RL environments, yielding up to a 400-fold reduction in the size of neural networks.

Related papers

• Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
  A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins. We employ inverse RL (IRL) to automatically learn reward functions without manual tuning. We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
  arXiv  Detail & Related papers  (2024-09-27T13:05:02Z)
• Trapezoidal Gradient Descent for Effective Reinforcement Learning in Spiking Networks [10.422381897413263]
  Spiking Neural Network (SNN) with their low energy consumption characteristics and performance have garnered widespread attention. To reduce the energy consumption of practical applications of reinforcement learning, researchers have successively proposed the Pop-SAN and MDC-SAN algorithms. We propose a trapezoidal approximation gradient method to replace the spike network, which not only preserves the original stable learning state but also enhances the model's adaptability and response sensitivity under various signal dynamics.
  arXiv  Detail & Related papers  (2024-06-19T13:56:22Z)
• Properties and Potential Applications of Random Functional-Linked Types of Neural Networks [81.56822938033119]
  Random functional-linked neural networks (RFLNNs) offer an alternative way of learning in deep structure. This paper gives some insights into the properties of RFLNNs from the viewpoints of frequency domain. We propose a method to generate a BLS network with better performance, and design an efficient algorithm for solving Poison's equation.
  arXiv  Detail & Related papers  (2023-04-03T13:25:22Z)
• A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learning [27.558298367330053]
  In this paper, we use learnable matrix multiplication to encode and decode spikes, improving the flexibility of the coders. We train the SNNs using the direct training method and use two different structures for online and offline RL algorithms. Experiments have revealed that our method achieves optimal performance with ultra-low latency and excellent energy efficiency.
  arXiv  Detail & Related papers  (2022-11-21T07:26:56Z)
• Single-Shot Pruning for Offline Reinforcement Learning [47.886329599997474]
  Deep Reinforcement Learning (RL) is a powerful framework for solving complex real-world problems. One way to tackle this problem is to prune neural networks leaving only the necessary parameters. We close the gap between RL and single-shot pruning techniques and present a general pruning approach to the Offline RL.
  arXiv  Detail & Related papers  (2021-12-31T18:10:02Z)
• A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement [0.7885276250519428]
  We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization based on the Power of Two Choices principle. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-05-14T10:04:17Z)
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference [56.24109486973292]
  We study the interplay between pruning and quantization during the training of neural networks for ultra low latency applications. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task.
  arXiv  Detail & Related papers  (2021-02-22T19:00:05Z)
• Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks [78.47459801017959]
  Sparsity can reduce the memory footprint of regular networks to fit mobile devices. We describe approaches to remove and add elements of neural networks, different training strategies to achieve model sparsity, and mechanisms to exploit sparsity in practice.
  arXiv  Detail & Related papers  (2021-01-31T22:48:50Z)
• ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks [63.91384986073851]
  We propose the autoencoder-based low-rank filter-sharing technique technique (ALF) ALF shows a reduction of 70% in network parameters, 61% in operations and 41% in execution time, with minimal loss in accuracy.
  arXiv  Detail & Related papers  (2020-07-27T09:01:22Z)
• Using Generative Adversarial Nets on Atari Games for Feature Extraction in Deep Reinforcement Learning [0.76146285961466]
  Deep Reinforcement Learning (DRL) has been successfully applied in several research domains such as robot navigation and automated video game playing. The main reason for this requirement is that sparse and delayed rewards do not provide an effective supervision for representation learning of deep neural networks. In this study, Proximal Policy Optimization (PPO) algorithm is augmented with Generative Adrial Networks (GANs) to increase the sample efficiency by enforcing the network to learn efficient representations without depending on sparse and delayed rewards as supervision.
  arXiv  Detail & Related papers  (2020-04-06T15:46:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.