An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

• URL: http://arxiv.org/abs/2101.04930v2
• Date: Wed, 10 Feb 2021 15:12:25 GMT
• Title: An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications
• Authors: Zhenpeng Chen and Huihan Yao and Yiling Lou and Yanbin Cao and Yuanqiang Liu and Haoyu Wang and Xuanzhe Liu
• Abstract summary: Mobile Deep Learning (DL) apps integrate DL models trained using large-scale data with DL programs. This paper presents the first comprehensive study on the deployment faults of mobile DL apps. We construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault types.
• Score: 7.58063287182615
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Learning (DL) is finding its way into a growing number of mobile software applications. These software applications, named as DL based mobile applications (abbreviated as mobile DL apps) integrate DL models trained using large-scale data with DL programs. A DL program encodes the structure of a desirable DL model and the process by which the model is trained using training data. Due to the increasing dependency of current mobile apps on DL, software engineering (SE) for mobile DL apps has become important. However, existing efforts in SE research community mainly focus on the development of DL models and extensively analyze faults in DL programs. In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied. Since mobile DL apps have been used by billions of end users daily for various purposes including for safety-critical scenarios, characterizing their deployment faults is of enormous importance. To fill the knowledge gap, this paper presents the first comprehensive study on the deployment faults of mobile DL apps. We identify 304 real deployment faults from Stack Overflow and GitHub, two commonly used data sources for studying software faults. Based on the identified faults, we construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault types. Furthermore, we suggest actionable implications and research avenues that could further facilitate the deployment of DL models on mobile devices.

Related papers

• Fault Localization in Deep Learning-based Software: A System-level Approach [12.546853096298175]
  We introduce FL4Deep, a system-level fault localization approach considering the entire Deep Learning development pipeline. In an evaluation using 100 faulty DL scripts, FL4Deep outperformed four previous approaches in terms of accuracy for three out of six DL-related faults.
  arXiv  Detail & Related papers  (2024-11-12T20:32:36Z)
• Science-Informed Deep Learning (ScIDL) With Applications to Wireless Communications [11.472232944923558]
  This article provides a tutorial on science-informed deep learning (ScIDL) ScIDL aims to integrate existing scientific knowledge with DL techniques to develop more powerful algorithms. We discuss both recent applications of ScIDL and potential future research directions in the field of wireless communications.
  arXiv  Detail & Related papers  (2024-06-29T02:35:39Z)
• Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? [60.50127555651554]
  Large Language Models (LLMs) show impressive results in numerous practical applications, but they lack essential safety features. This makes them vulnerable to manipulations such as indirect prompt injections and generally unsuitable for safety-critical tasks. We introduce a formal measure for instruction-data separation and an empirical variant that is calculable from a model's outputs.
  arXiv  Detail & Related papers  (2024-03-11T15:48:56Z)
• Beyond the Model: Data Pre-processing Attack to Deep Learning Models in Android Apps [3.2307366446033945]
  We introduce a data processing-based attack against real-world deep learning (DL) apps. Our attack could influence the performance and latency of the model without affecting the operation of a DL app. Among 320 apps utilizing MLkit, we find that 81.56% of them can be successfully attacked.
  arXiv  Detail & Related papers  (2023-05-06T07:35:39Z)
• Learnware: Small Models Do Big [69.88234743773113]
  The prevailing big model paradigm, which has achieved impressive results in natural language processing and computer vision applications, has not yet addressed those issues, whereas becoming a serious source of carbon emissions. This article offers an overview of the learnware paradigm, which attempts to enable users not need to build machine learning models from scratch, with the hope of reusing small models to do things even beyond their original purposes.
  arXiv  Detail & Related papers  (2022-10-07T15:55:52Z)
• Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning [46.924429562606086]
  We propose to integrate the abundance of scientific knowledge sources (SKS) with the deep learning (DL) training process. Existing knowledge integration approaches are limited to using differentiable knowledge source to be compatible with first-order DL training paradigm. We show that proposed scheme is able to effectively integrate scientific knowledge with DL training and is able to outperform purely data-driven model for data-limited scientific applications.
  arXiv  Detail & Related papers  (2022-06-04T17:52:42Z)
• A Comprehensive Benchmark of Deep Learning Libraries on Mobile Devices [12.342282138576348]
  We build a benchmark that includes 6 representative DL libs and 15 diversified DL models. We then perform extensive experiments on 10 mobile devices, which help reveal a complete landscape of the current mobile DL libs ecosystem. We find that the best-performing DL lib is severely fragmented across different models and hardware.
  arXiv  Detail & Related papers  (2022-02-14T07:00:31Z)
• Towards Training Reproducible Deep Learning Models [26.547756923322126]
  Deep Learning (DL) models are challenging to be reproduced due to issues like randomness in the software and non-determinism in the hardware. This paper proposes a systematic approach to training reproducible DL models. Case study results show our approach can successfully reproduce six open source and one commercial DL models.
  arXiv  Detail & Related papers  (2022-02-04T18:14:39Z)
• A First Look at Class Incremental Learning in Deep Learning Mobile Traffic Classification [68.11005070665364]
  We explore Incremental Learning (IL) techniques to add new classes to models without a full retraining, hence speeding up model's updates cycle. We consider iCarl, a state of the art IL method, and MIRAGE-2019, a public dataset with traffic from 40 Android apps. Despite our analysis reveals their infancy, IL techniques are a promising research area on the roadmap towards automated DL-based traffic analysis systems.
  arXiv  Detail & Related papers  (2021-07-09T14:28:16Z)
• Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications [72.02908263225919]
  We share our experience on a commercial-grade DL traffic classification engine. We identify known applications from encrypted traffic, as well as unknown zero-day applications. We propose a novel technique, tailored for DL models, that is significantly more accurate and light-weight than the state of the art.
  arXiv  Detail & Related papers  (2021-04-07T15:21:22Z)
• A Survey of Deep Active Learning [54.376820959917005]
  Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples. Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters. Deep active learning (DAL) has emerged.
  arXiv  Detail & Related papers  (2020-08-30T04:28:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.