A Survey of Deep Learning Library Testing Methods

• URL: http://arxiv.org/abs/2404.17871v2
• Date: Mon, 25 Nov 2024 06:51:06 GMT
• Title: A Survey of Deep Learning Library Testing Methods
• Authors: Xiaoyu Zhang, Weipeng Jiang, Chao Shen, Qi Li, Qian Wang, Chenhao Lin, Xiaohong Guan,
• Abstract summary: Deep learning (DL) libraries undertake the underlying optimization and computation. DL libraries are not immune to bugs, which can pose serious threats to users' personal property and safety. This paper provides an overview of the testing research related to various DL libraries.
• Score: 33.62859142913532
• License:
• Abstract: In recent years, software systems powered by deep learning (DL) techniques have significantly facilitated people's lives in many aspects. As the backbone of these DL systems, various DL libraries undertake the underlying optimization and computation. However, like traditional software, DL libraries are not immune to bugs, which can pose serious threats to users' personal property and safety. Studying the characteristics of DL libraries, their associated bugs, and the corresponding testing methods is crucial for enhancing the security of DL systems and advancing the widespread application of DL technology. This paper provides an overview of the testing research related to various DL libraries, discusses the strengths and weaknesses of existing methods, and provides guidance and reference for the application of the DL library. This paper first introduces the workflow of DL underlying libraries and the characteristics of three kinds of DL libraries involved, namely DL framework, DL compiler, and DL hardware library. It then provides definitions for DL underlying library bugs and testing. Additionally, this paper summarizes the existing testing methods and tools tailored to these DL libraries separately and analyzes their effectiveness and limitations. It also discusses the existing challenges of DL library testing and outlines potential directions for future research.

Related papers

• A Tale of Two DL Cities: When Library Tests Meet Compiler [12.751626834965231]
  We propose OPERA to extract domain knowledge from the test inputs for DL libraries. OPERA constructs diverse tests from the various test inputs for DL libraries. It incorporates a diversity-based test prioritization strategy to migrate and execute those test inputs.
  arXiv  Detail & Related papers  (2024-07-23T16:35:45Z)
• Katakomba: Tools and Benchmarks for Data-Driven NetHack [52.0035089982277]
  NetHack is known as the frontier of reinforcement learning research. We argue that there are three major obstacles for adoption: resource-wise, implementation-wise, and benchmark-wise. We develop an open-source library that provides workflow fundamentals familiar to the offline reinforcement learning community.
  arXiv  Detail & Related papers  (2023-06-14T22:50:25Z)
• SequeL: A Continual Learning Library in PyTorch and JAX [50.33956216274694]
  SequeL is a library for Continual Learning that supports both PyTorch and JAX frameworks. It provides a unified interface for a wide range of Continual Learning algorithms, including regularization-based approaches, replay-based approaches, and hybrid approaches. We release SequeL as an open-source library, enabling researchers and developers to easily experiment and extend the library for their own purposes.
  arXiv  Detail & Related papers  (2023-04-21T10:00:22Z)
• MEMO: Coverage-guided Model Generation For Deep Learning Library Testing [11.263121366956726]
  A few techniques have been proposed to test deep learning (DL) libraries by generating DL models as test inputs. But the test effectiveness of these techniques is constrained by the diversity of generated DL models. We propose MEMO to efficiently generate diverse DL models by exploring layer types, layer pairs, and layer parameters.
  arXiv  Detail & Related papers  (2022-08-02T14:53:02Z)
• A Comprehensive Benchmark of Deep Learning Libraries on Mobile Devices [12.342282138576348]
  We build a benchmark that includes 6 representative DL libs and 15 diversified DL models. We then perform extensive experiments on 10 mobile devices, which help reveal a complete landscape of the current mobile DL libs ecosystem. We find that the best-performing DL lib is severely fragmented across different models and hardware.
  arXiv  Detail & Related papers  (2022-02-14T07:00:31Z)
• Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning Workloads [86.62083829086393]
  This work introduces the Processing Primitives (TPP), a programming abstraction striving for efficient, portable implementation of Deep Learning-workloads with high-productivity. TPPs define a compact, yet versatile set of 2D-tensor operators (or a virtual ISA), which can be utilized as building-blocks to construct complex operators on high-dimensional tensors. We demonstrate the efficacy of our approach using standalone kernels and end-to-end DL-workloads expressed entirely via TPPs that outperform state-of-the-art implementations on multiple platforms.
  arXiv  Detail & Related papers  (2021-04-12T18:35:49Z)
• An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications [7.58063287182615]
  Mobile Deep Learning (DL) apps integrate DL models trained using large-scale data with DL programs. This paper presents the first comprehensive study on the deployment faults of mobile DL apps. We construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault types.
  arXiv  Detail & Related papers  (2021-01-13T08:19:50Z)
• A Survey of Deep Active Learning [54.376820959917005]
  Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples. Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters. Deep active learning (DAL) has emerged.
  arXiv  Detail & Related papers  (2020-08-30T04:28:31Z)
• Automated Reasoning in Temporal DL-Lite [65.9825143048822]
  This paper investigates the feasibility of automated reasoning over temporal DL-Lite (TDL-Lite) knowledge bases (KBs) We test the usage of off-theshelf reasoners to check satisfiability of TDL-Lite KBs. In an effort to make the usage of TDL-Lite KBs a reality, we present a fully fledged tool with a graphical interface to design them.
  arXiv  Detail & Related papers  (2020-08-17T16:40:27Z)
• The Deep Learning Compiler: A Comprehensive Survey [16.19025439622745]
  We perform a comprehensive survey of existing DL compilers by dissecting the commonly adopted design in details. Specifically, we provide a comprehensive comparison among existing DL compilers from various aspects.
  arXiv  Detail & Related papers  (2020-02-06T07:29:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.