Unsolved Problems in ML Safety

• URL: http://arxiv.org/abs/2109.13916v1
• Date: Tue, 28 Sep 2021 17:59:36 GMT
• Title: Unsolved Problems in ML Safety
• Authors: Dan Hendrycks and Nicholas Carlini and John Schulman and Jacob Steinhardt
• Abstract summary: We present four problems ready for research, namely withstanding hazards, identifying hazards, steering ML systems, and reducing risks to how ML systems are handled. We clarify each problem's motivation and provide concrete research directions.
• Score: 45.82027272958549
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technologies, safety for ML should be a leading research priority. In response to emerging safety challenges in ML, such as those introduced by recent large-scale models, we provide a new roadmap for ML Safety and refine the technical problems that the field needs to address. We present four problems ready for research, namely withstanding hazards ("Robustness"), identifying hazards ("Monitoring"), steering ML systems ("Alignment"), and reducing risks to how ML systems are handled ("External Safety"). Throughout, we clarify each problem's motivation and provide concrete research directions.

Related papers

• SLM as Guardian: Pioneering AI Safety with Small Language Models [6.799423428734095]
  Internalizing safeguard features into larger models brought challenges of higher training cost and unintended degradation of helpfulness. In this paper, we leverage a smaller LLM for both harmful query detection and safeguard response generation. We demonstrate the effectiveness of our approach, providing on par or surpassing harmful query detection and safeguard response performance compared to the publicly available LLMs.
  arXiv  Detail & Related papers  (2024-05-30T08:03:15Z)
• Highlighting the Safety Concerns of Deploying LLMs/VLMs in Robotics [54.57914943017522]
  We highlight the critical issues of robustness and safety associated with integrating large language models (LLMs) and vision-language models (VLMs) into robotics applications.
  arXiv  Detail & Related papers  (2024-02-15T22:01:45Z)
• Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review [51.31851488650698]
  Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid. adversarial distortion injected into the power signal will greatly affect the system's normal control and operation. It is imperative to conduct vulnerability assessment for MLsgAPPs applied in the context of safety-critical power systems.
  arXiv  Detail & Related papers  (2023-08-30T03:29:26Z)
• Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey [2.8348950186890467]
  Machine learning (ML) is finding its way into safety-critical systems (SCS) Current safety standards and practice were not designed to cope with ML techniques.
  arXiv  Detail & Related papers  (2021-11-29T04:53:39Z)
• Practical Machine Learning Safety: A Survey and Primer [81.73857913779534]
  Open-world deployment of Machine Learning algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities. New models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. Our organization maps state-of-the-art ML techniques to safety strategies in order to enhance the dependability of the ML algorithm from different aspects.
  arXiv  Detail & Related papers  (2021-06-09T05:56:42Z)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety [54.478842696269304]
  The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. Our paper addresses both machine learning experts and safety engineers.
  arXiv  Detail & Related papers  (2021-04-29T09:54:54Z)
• White Paper Machine Learning in Certified Systems [70.24215483154184]
  DEEL Project set-up the ML Certification 3 Workgroup (WG) set-up by the Institut de Recherche Technologique Saint Exup'ery de Toulouse (IRT)
  arXiv  Detail & Related papers  (2021-03-18T21:14:30Z)
• Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead [24.60052335548398]
  Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT) They are vulnerable to various security and reliability threats, at both hardware and software levels, that compromise their accuracy. This paper summarizes the prominent vulnerabilities of modern ML systems, highlights successful defenses and mitigation techniques against these vulnerabilities.
  arXiv  Detail & Related papers  (2021-01-04T20:06:56Z)
• Machine Learning (In) Security: A Stream of Problems [17.471312325933244]
  We identify, detail, and discuss the main challenges in the correct application of Machine Learning techniques to cybersecurity data. We evaluate how concept drift, evolution, delayed labels, and adversarial ML impact the existing solutions. We present how existing solutions may fail under certain circumstances, and propose mitigations to them.
  arXiv  Detail & Related papers  (2020-10-30T03:40:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.