SoK: Evaluations in Industrial Intrusion Detection Research
- URL: http://arxiv.org/abs/2311.02929v1
- Date: Mon, 6 Nov 2023 07:49:58 GMT
- Title: SoK: Evaluations in Industrial Intrusion Detection Research
- Authors: Olav Lamberts, Konrad Wolsing, Eric Wagner, Jan Pennekamp, Jan Bauer, Klaus Wehrle, Martin Henze,
- Abstract summary: Industrial intrusion detection systems strive to timely uncover even the most sophisticated breaches.
Due to its criticality for society, this fast-growing field attracts researchers from diverse backgrounds.
Our analysis of 609 publications shows that the rapid growth of this research field has positive and negative consequences.
- Score: 8.356036431147889
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Industrial systems are increasingly threatened by cyberattacks with potentially disastrous consequences. To counter such attacks, industrial intrusion detection systems strive to timely uncover even the most sophisticated breaches. Due to its criticality for society, this fast-growing field attracts researchers from diverse backgrounds, resulting in 130 new detection approaches in 2021 alone. This huge momentum facilitates the exploration of diverse promising paths but likewise risks fragmenting the research landscape and burying promising progress. Consequently, it needs sound and comprehensible evaluations to mitigate this risk and catalyze efforts into sustainable scientific progress with real-world applicability. In this paper, we therefore systematically analyze the evaluation methodologies of this field to understand the current state of industrial intrusion detection research. Our analysis of 609 publications shows that the rapid growth of this research field has positive and negative consequences. While we observe an increased use of public datasets, publications still only evaluate 1.3 datasets on average, and frequently used benchmarking metrics are ambiguous. At the same time, the adoption of newly developed benchmarking metrics sees little advancement. Finally, our systematic analysis enables us to provide actionable recommendations for all actors involved and thus bring the entire research field forward.
Related papers
- A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods.
The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.
We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
arXiv Detail & Related papers (2024-06-05T13:40:07Z) - Temporal Analysis of World Disaster Risk:A Machine Learning Approach to
Cluster Dynamics [0.0]
This paper assesses the impact of efforts considered to mitigate risk and create safe environments on a global scale.
Using the World Risk Index, we conduct a temporal analysis of global disaster risk dynamics from 2011 to 2021.
arXiv Detail & Related papers (2024-01-10T08:50:53Z) - Navigating the Concurrency Landscape: A Survey of Race Condition Vulnerability Detectors [0.12289361708127873]
This paper focuses on the realm of race condition bug detectors.
We systematically categorize these detectors based on the diverse methodologies they employ.
We shed light on the application of fuzzing techniques in the detection of race condition vulnerabilities.
arXiv Detail & Related papers (2023-12-22T07:05:39Z) - A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual
Learning [76.47138162283714]
Forgetting refers to the loss or deterioration of previously acquired information or knowledge.
Forgetting is a prevalent phenomenon observed in various other research domains within deep learning.
Survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases.
arXiv Detail & Related papers (2023-07-16T16:27:58Z) - Industry Risk Assessment via Hierarchical Financial Data Using Stock Market Sentiment Indicators [0.9463895540925061]
This paper presents an approach to analyzing industry trends leveraging real-time stock market data and generative small language models (SLMs)
One of the key challenges lies in the inherent noise in raw data, which can compromise the precision of statistical analyses.
We propose a dual-pronged approach to industry trend analysis: explicit and implicit analysis.
arXiv Detail & Related papers (2023-03-05T16:17:56Z) - Fairness in Recommender Systems: Research Landscape and Future
Directions [119.67643184567623]
We review the concepts and notions of fairness that were put forward in the area in the recent past.
We present an overview of how research in this field is currently operationalized.
Overall, our analysis of recent works points to certain research gaps.
arXiv Detail & Related papers (2022-05-23T08:34:25Z) - A Survey on Unsupervised Anomaly Detection Algorithms for Industrial
Images [2.4976719861186845]
In line with the development of Industry 4.0, surface defect detection/anomaly detection becomes a topical subject in the industry field.
Unsupervised learning has great potential in tackling the above disadvantages for visual industrial anomaly detection.
arXiv Detail & Related papers (2022-04-24T01:38:18Z) - No Need to Know Physics: Resilience of Process-based Model-free Anomaly
Detection for Industrial Control Systems [95.54151664013011]
We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system.
We analyze four anomaly detectors published at top security conferences.
arXiv Detail & Related papers (2020-12-07T11:02:44Z) - Anomalous Example Detection in Deep Learning: A Survey [98.2295889723002]
This survey tries to provide a structured and comprehensive overview of the research on anomaly detection for Deep Learning applications.
We provide a taxonomy for existing techniques based on their underlying assumptions and adopted approaches.
We highlight the unsolved research challenges while applying anomaly detection techniques in DL systems and present some high-impact future research directions.
arXiv Detail & Related papers (2020-03-16T02:47:23Z) - Survey of Network Intrusion Detection Methods from the Perspective of
the Knowledge Discovery in Databases Process [63.75363908696257]
We review the methods that have been applied to network data with the purpose of developing an intrusion detector.
We discuss the techniques used for the capture, preparation and transformation of the data, as well as, the data mining and evaluation methods.
As a result of this literature review, we investigate some open issues which will need to be considered for further research in the area of network security.
arXiv Detail & Related papers (2020-01-27T11:21:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.