Machine Learning (In) Security: A Stream of Problems
- URL: http://arxiv.org/abs/2010.16045v2
- Date: Mon, 4 Sep 2023 17:05:32 GMT
- Title: Machine Learning (In) Security: A Stream of Problems
- Authors: Fabr\'icio Ceschin and Marcus Botacin and Albert Bifet and Bernhard
Pfahringer and Luiz S. Oliveira and Heitor Murilo Gomes and Andr\'e Gr\'egio
- Abstract summary: We identify, detail, and discuss the main challenges in the correct application of Machine Learning techniques to cybersecurity data.
We evaluate how concept drift, evolution, delayed labels, and adversarial ML impact the existing solutions.
We present how existing solutions may fail under certain circumstances, and propose mitigations to them.
- Score: 17.471312325933244
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Machine Learning (ML) has been widely applied to cybersecurity and is
considered state-of-the-art for solving many of the open issues in that field.
However, it is very difficult to evaluate how good the produced solutions are,
since the challenges faced in security may not appear in other areas. One of
these challenges is the concept drift, which increases the existing arms race
between attackers and defenders: malicious actors can always create novel
threats to overcome the defense solutions, which may not consider them in some
approaches. Due to this, it is essential to know how to properly build and
evaluate an ML-based security solution. In this paper, we identify, detail, and
discuss the main challenges in the correct application of ML techniques to
cybersecurity data. We evaluate how concept drift, evolution, delayed labels,
and adversarial ML impact the existing solutions. Moreover, we address how
issues related to data collection affect the quality of the results presented
in the security literature, showing that new strategies are needed to improve
current solutions. Finally, we present how existing solutions may fail under
certain circumstances, and propose mitigations to them, presenting a novel
checklist to help the development of future ML solutions for cybersecurity.
Related papers
- Multimodal Situational Safety [73.63981779844916]
We present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety.
For an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context.
We develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs.
arXiv Detail & Related papers (2024-10-08T16:16:07Z) - Safety in Graph Machine Learning: Threats and Safeguards [84.26643884225834]
Despite their societal benefits, recent research highlights significant safety concerns associated with the widespread use of Graph ML models.
Lacking safety-focused designs, these models can produce unreliable predictions, demonstrate poor generalizability, and compromise data confidentiality.
In high-stakes scenarios such as financial fraud detection, these vulnerabilities could jeopardize both individuals and society at large.
arXiv Detail & Related papers (2024-05-17T18:11:11Z) - Threats, Attacks, and Defenses in Machine Unlearning: A Survey [14.03428437751312]
Machine Unlearning (MU) has recently gained considerable attention due to its potential to achieve Safe AI.
This survey aims to fill the gap between the extensive number of studies on threats, attacks, and defenses in machine unlearning.
arXiv Detail & Related papers (2024-03-20T15:40:18Z) - New Challenges in Reinforcement Learning: A Survey of Security and
Privacy [26.706957408693363]
Reinforcement learning (RL) is one of the most important branches of AI.
RL has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics.
Some of these applications and systems have been shown to be vulnerable to security or privacy attacks.
arXiv Detail & Related papers (2022-12-31T12:30:43Z) - Security for Machine Learning-based Software Systems: a survey of
threats, practices and challenges [0.76146285961466]
How to securely develop the machine learning-based modern software systems (MLBSS) remains a big challenge.
latent vulnerabilities and privacy issues exposed to external users and attackers will be largely neglected and hard to be identified.
We consider that security for machine learning-based software systems may arise from inherent system defects or external adversarial attacks.
arXiv Detail & Related papers (2022-01-12T23:20:25Z) - Unsolved Problems in ML Safety [45.82027272958549]
We present four problems ready for research, namely withstanding hazards, identifying hazards, steering ML systems, and reducing risks to how ML systems are handled.
We clarify each problem's motivation and provide concrete research directions.
arXiv Detail & Related papers (2021-09-28T17:59:36Z) - Inspect, Understand, Overcome: A Survey of Practical Methods for AI
Safety [54.478842696269304]
The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings.
In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged.
Our paper addresses both machine learning experts and safety engineers.
arXiv Detail & Related papers (2021-04-29T09:54:54Z) - Understanding the Usability Challenges of Machine Learning In
High-Stakes Decision Making [67.72855777115772]
Machine learning (ML) is being applied to a diverse and ever-growing set of domains.
In many cases, domain experts -- who often have no expertise in ML or data science -- are asked to use ML predictions to make high-stakes decisions.
We investigate the ML usability challenges present in the domain of child welfare screening through a series of collaborations with child welfare screeners.
arXiv Detail & Related papers (2021-03-02T22:50:45Z) - Robust Machine Learning Systems: Challenges, Current Trends,
Perspectives, and the Road Ahead [24.60052335548398]
Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT)
They are vulnerable to various security and reliability threats, at both hardware and software levels, that compromise their accuracy.
This paper summarizes the prominent vulnerabilities of modern ML systems, highlights successful defenses and mitigation techniques against these vulnerabilities.
arXiv Detail & Related papers (2021-01-04T20:06:56Z) - Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance.
We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems.
We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
arXiv Detail & Related papers (2020-10-19T13:09:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.