Reusing Deep Learning Models: Challenges and Directions in Software Engineering

• URL: http://arxiv.org/abs/2404.16688v1
• Date: Thu, 25 Apr 2024 15:42:10 GMT
• Title: Reusing Deep Learning Models: Challenges and Directions in Software Engineering
• Authors: James C. Davis, Purvish Jajal, Wenxin Jiang, Taylor R. Schorlemmer, Nicholas Synovic, George K. Thiruvathukal,
• Abstract summary: Deep neural networks (DNNs) achieve state-of-the-art performance in many areas. DNNs are expensive to develop, both in intellectual effort and computational costs. This vision paper describes challenges in current approaches to DNN re-use.
• Score: 3.733306025181894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks (DNNs) achieve state-of-the-art performance in many areas, including computer vision, system configuration, and question-answering. However, DNNs are expensive to develop, both in intellectual effort (e.g., devising new architectures) and computational costs (e.g., training). Reusing DNNs is a promising direction to amortize costs within a company and across the computing industry. As with any new technology, however, there are many challenges in reusing DNNs. These challenges include both missing technical capabilities and missing engineering practices. This vision paper describes challenges in current approaches to DNN re-use. We summarize studies of re-use failures across the spectrum of re-use techniques, including conceptual (e.g., reusing based on a research paper), adaptation (e.g., re-using by building on an existing implementation), and deployment (e.g., direct re-use on a new device). We outline possible advances that would improve each kind of re-use.

Related papers

• Survey on Computer Vision Techniques for Internet-of-Things Devices [0.0]
  Deep neural networks (DNNs) are state-of-the-art techniques for solving computer vision problems. DNNs require billions of parameters and operations to achieve state-of-the-art results. This requirement makes DNNs extremely compute, memory, and energy-hungry, and consequently difficult to deploy on small battery-powered Internet-of-Things (IoT) devices with limited computing resources.
  arXiv  Detail & Related papers  (2023-08-02T03:41:24Z)
• Transferability of Convolutional Neural Networks in Stationary Learning Tasks [96.00428692404354]
  We introduce a novel framework for efficient training of convolutional neural networks (CNNs) for large-scale spatial problems. We show that a CNN trained on small windows of such signals achieves a nearly performance on much larger windows without retraining. Our results show that the CNN is able to tackle problems with many hundreds of agents after being trained with fewer than ten.
  arXiv  Detail & Related papers  (2023-07-21T13:51:45Z)
• Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
  Deep learning has led to a rising demand for neural network architecture engineering. neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually. NAS has become applicable to a much wider range of problems in computer vision.
  arXiv  Detail & Related papers  (2022-02-15T08:06:50Z)
• ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks [13.661704974188872]
  We propose a novel repairing direction for deep neural networks (DNNs) at the block level. We propose adversarial-aware spectrum analysis for vulnerable block localization. We also propose the architecture-oriented search-based repairing that relaxes the targeted block to a continuous repairing search space.
  arXiv  Detail & Related papers  (2021-11-26T06:35:15Z)
• Pruning and Slicing Neural Networks using Formal Verification [0.2538209532048866]
  Deep neural networks (DNNs) play an increasingly important role in various computer systems. In order to create these networks, engineers typically specify a desired topology, and then use an automated training algorithm to select the network's weights. Here, we propose to address this challenge by harnessing recent advances in DNN verification.
  arXiv  Detail & Related papers  (2021-05-28T07:53:50Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges [0.0]
  A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN) Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs.
  arXiv  Detail & Related papers  (2020-06-09T17:28:25Z)
• Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G Networks [84.2155885234293]
  We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC. To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
  arXiv  Detail & Related papers  (2020-02-22T14:38:11Z)
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets [83.12737997548645]
  Skip connections are an essential component of current state-of-the-art deep neural networks (DNNs) Use of skip connections allows easier generation of highly transferable adversarial examples. We conduct comprehensive transfer attacks against state-of-the-art DNNs including ResNets, DenseNets, Inceptions, Inception-ResNet, Squeeze-and-Excitation Network (SENet)
  arXiv  Detail & Related papers  (2020-02-14T12:09:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.