Software Testing for Machine Learning

• URL: http://arxiv.org/abs/2205.00210v1
• Date: Sat, 30 Apr 2022 08:47:10 GMT
• Title: Software Testing for Machine Learning
• Authors: Dusica Marijan and Arnaud Gotlieb
• Abstract summary: Machine learning has shown to be susceptible to deception, leading to errors and even fatal failures. This circumstance calls into question the widespread use of machine learning, especially in safety-critical applications. This summary talk discusses the current state-of-the-art of software testing for machine learning.
• Score: 13.021014899410684
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Machine learning has become prevalent across a wide variety of applications. Unfortunately, machine learning has also shown to be susceptible to deception, leading to errors, and even fatal failures. This circumstance calls into question the widespread use of machine learning, especially in safety-critical applications, unless we are able to assure its correctness and trustworthiness properties. Software verification and testing are established technique for assuring such properties, for example by detecting errors. However, software testing challenges for machine learning are vast and profuse - yet critical to address. This summary talk discusses the current state-of-the-art of software testing for machine learning. More specifically, it discusses six key challenge areas for software testing of machine learning systems, examines current approaches to these challenges and highlights their limitations. The paper provides a research agenda with elaborated directions for making progress toward advancing the state-of-the-art on testing of machine learning.

Related papers

• Machine Unlearning: A Survey [56.79152190680552]
  A special need has arisen where, due to privacy, usability, and/or the right to be forgotten, information about some specific samples needs to be removed from a model, called machine unlearning. This emerging technology has drawn significant interest from both academics and industry due to its innovation and practicality. No study has analyzed this complex topic or compared the feasibility of existing unlearning solutions in different kinds of scenarios. The survey concludes by highlighting some of the outstanding issues with unlearning techniques, along with some feasible directions for new research opportunities.
  arXiv  Detail & Related papers  (2023-06-06T10:18:36Z)
• Modelling Concurrency Bugs Using Machine Learning [0.0]
  This project aims to compare both common and recent machine learning approaches. We define a synthetic dataset that we generate with the scope of simulating real-life (concurrent) programs. We formulate hypotheses about fundamental limits of various machine learning model types.
  arXiv  Detail & Related papers  (2023-05-08T17:30:24Z)
• Security for Machine Learning-based Software Systems: a survey of threats, practices and challenges [0.76146285961466]
  How to securely develop the machine learning-based modern software systems (MLBSS) remains a big challenge. latent vulnerabilities and privacy issues exposed to external users and attackers will be largely neglected and hard to be identified. We consider that security for machine learning-based software systems may arise from inherent system defects or external adversarial attacks.
  arXiv  Detail & Related papers  (2022-01-12T23:20:25Z)
• Knowledge as Invariance -- History and Perspectives of Knowledge-augmented Machine Learning [69.99522650448213]
  Research in machine learning is at a turning point. Research interests are shifting away from increasing the performance of highly parameterized models to exceedingly specific tasks. This white paper provides an introduction and discussion of this emerging field in machine learning research.
  arXiv  Detail & Related papers  (2020-12-21T15:07:19Z)
• Challenges in Deploying Machine Learning: a Survey of Case Studies [11.028123436097616]
  This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process.
  arXiv  Detail & Related papers  (2020-11-18T16:20:28Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)
• Smoke Testing for Machine Learning: Simple Tests to Discover Severe Defects [7.081604594416339]
  We try to determine generic and simple smoke tests that can be used to assert that basic functions can be executed without crashing. We were able to find bugs in all three machine learning libraries that we tested and severe bugs in two of the three libraries.
  arXiv  Detail & Related papers  (2020-09-03T08:54:43Z)
• A Survey of Machine Learning Methods and Challenges for Windows Malware Classification [43.4550536920809]
  Survey aims to be useful both to cybersecurity practitioners who wish to learn more about how machine learning can be applied to the malware problem, and to give data scientists the necessary background into the challenges in this uniquely complicated space.
  arXiv  Detail & Related papers  (2020-06-15T17:46:12Z)
• Machine Learning for Software Engineering: A Systematic Mapping [73.30245214374027]
  The software development industry is rapidly adopting machine learning for transitioning modern day software systems towards highly intelligent and self-learning systems. No comprehensive study exists that explores the current state-of-the-art on the adoption of machine learning across software engineering life cycle stages. This study introduces a machine learning for software engineering (MLSE) taxonomy classifying the state-of-the-art machine learning techniques according to their applicability to various software engineering life cycle stages.
  arXiv  Detail & Related papers  (2020-05-27T11:56:56Z)
• Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology [53.063411515511056]
  We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
  arXiv  Detail & Related papers  (2020-03-11T08:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.