Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision

• URL: http://arxiv.org/abs/2303.07476v2
• Date: Fri, 25 Aug 2023 18:58:49 GMT
• Title: Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision
• Authors: Wenxin Jiang, Vishnu Banna, Naveen Vivek, Abhinav Goel, Nicholas Synovic, George K. Thiruvathukal, James C. Davis
• Abstract summary: Many engineering organizations are reimplementing and extending deep neural networks from the research community. Deep learning model reengineering is challenging for reasons including under-documented reference models, changing requirements, and the cost of implementation and testing. Our study is focused on reengineering activities from a "process" view, and focuses on engineers specifically engaged in the reengineering process.
• Score: 3.510650664260664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many engineering organizations are reimplementing and extending deep neural networks from the research community. We describe this process as deep learning model reengineering. Deep learning model reengineering - reusing, reproducing, adapting, and enhancing state-of-the-art deep learning approaches - is challenging for reasons including under-documented reference models, changing requirements, and the cost of implementation and testing. In addition, individual engineers may lack expertise in software engineering, yet teams must apply knowledge of software engineering and deep learning to succeed. Prior work has examined on DL systems from a "product" view, examining defects from projects regardless of the engineers' purpose. Our study is focused on reengineering activities from a "process" view, and focuses on engineers specifically engaged in the reengineering process. Our goal is to understand the characteristics and challenges of deep learning model reengineering. We conducted a case study of this phenomenon, focusing on the context of computer vision. Our results draw from two data sources: defects reported in open-source reeengineering projects, and interviews conducted with open-source project contributors and the leaders of a reengineering team. Our results describe how deep learning-based computer vision techniques are reengineered, analyze the distribution of defects in this process, and discuss challenges and practices. Integrating our quantitative and qualitative data, we proposed a novel reengineering workflow. Our findings inform several future directions, including: measuring additional unknown aspects of model reengineering; standardizing engineering practices to facilitate reengineering; and developing tools to support model reengineering and model reuse.

Related papers

• Data Publishing in Mechanics and Dynamics: Challenges, Guidelines, and Examples from Engineering Design [4.065325208853021]
  This article analyzes the value and challenges of data publishing in mechanics and dynamics. It shows that the latter raise also challenges and considerations not typical in fields where data-driven methods have been booming originally.
  arXiv  Detail & Related papers  (2024-10-07T18:26:05Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design [5.268919870502001]
  This paper presents a comprehensive evaluation of vision-language models (VLMs) across a spectrum of engineering design tasks. Specifically in this paper, we assess the capabilities of two VLMs, GPT-4V and LLaVA 1.6 34B, in design tasks such as sketch similarity analysis, CAD generation, topology optimization, manufacturability assessment, and engineering textbook problems.
  arXiv  Detail & Related papers  (2023-11-21T15:20:48Z)
• Learnware: Small Models Do Big [69.88234743773113]
  The prevailing big model paradigm, which has achieved impressive results in natural language processing and computer vision applications, has not yet addressed those issues, whereas becoming a serious source of carbon emissions. This article offers an overview of the learnware paradigm, which attempts to enable users not need to build machine learning models from scratch, with the hope of reusing small models to do things even beyond their original purposes.
  arXiv  Detail & Related papers  (2022-10-07T15:55:52Z)
• Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning [65.268245109828]
  In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models. Deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning.
  arXiv  Detail & Related papers  (2022-02-22T02:33:54Z)
• Deep Generative Models in Engineering Design: A Review [1.933681537640272]
  We present a review and analysis of Deep Generative Learning models in engineering design. Recent DGMs have shown promising results in design applications like structural optimization, materials design, and shape synthesis.
  arXiv  Detail & Related papers  (2021-10-21T02:50:10Z)
• An Experience Report on Machine Learning Reproducibility: Guidance for Practitioners and TensorFlow Model Garden Contributors [1.177923904173852]
  This report is to define a process for reproducing a state-of-the-art machine learning model at a level of quality suitable for inclusion in the Model Garden. We report on our experiences implementing the YOLO model family with a team of 26 student researchers, share the tools we developed, and describe the lessons we learned along the way.
  arXiv  Detail & Related papers  (2021-07-02T04:32:18Z)
• Knowledge Distillation: A Survey [87.51063304509067]
  Deep neural networks have been successful in both industry and academia, especially for computer vision tasks. It is a challenge to deploy these cumbersome deep models on devices with limited resources. Knowledge distillation effectively learns a small student model from a large teacher model.
  arXiv  Detail & Related papers  (2020-06-09T21:47:17Z)
• Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology [53.063411515511056]
  We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
  arXiv  Detail & Related papers  (2020-03-11T08:25:49Z)
• Engineering AI Systems: A Research Agenda [9.84673609667263]
  We provide a conceptualization of the typical evolution patterns that companies experience when employing machine learning. The main contribution of the paper is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions.
  arXiv  Detail & Related papers  (2020-01-16T20:29:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.