SKTR: Trace Recovery from Stochastically Known Logs
- URL: http://arxiv.org/abs/2206.12672v3
- Date: Fri, 28 Jul 2023 04:35:15 GMT
- Title: SKTR: Trace Recovery from Stochastically Known Logs
- Authors: Eli Bogdanov, Izack Cohen, Avigdor Gal
- Abstract summary: Developments in machine learning together with the increasing usage of sensor data challenge the reliance on deterministic logs.
In this work we formulate the task of generating a deterministic log fromally known logs that is as faithful to reality as possible.
An effective trace recovery algorithm would be a powerful aid for maintaining credible process mining tools for uncertain settings.
- Score: 7.882975068446842
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Developments in machine learning together with the increasing usage of sensor
data challenge the reliance on deterministic logs, requiring new process mining
solutions for uncertain, and in particular stochastically known, logs. In this
work we formulate {trace recovery}, the task of generating a deterministic log
from stochastically known logs that is as faithful to reality as possible. An
effective trace recovery algorithm would be a powerful aid for maintaining
credible process mining tools for uncertain settings. We propose an algorithmic
framework for this task that recovers the best alignment between a
stochastically known log and a process model, with three innovative features.
Our algorithm, SKTR, 1) handles both Markovian and non-Markovian processes; 2)
offers a quality-based balance between a process model and a log, depending on
the available process information, sensor quality, and machine learning
predictiveness power; and 3) offers a novel use of a synchronous product
multigraph to create the log. An empirical analysis using five publicly
available datasets, three of which use predictive models over standard video
capturing benchmarks, shows an average relative accuracy improvement of more
than 10 over a common baseline.
Related papers
- RAPID: Training-free Retrieval-based Log Anomaly Detection with PLM
considering Token-level information [7.861095039299132]
The need for log anomaly detection is growing, especially in real-world applications.
Traditional deep learning-based anomaly detection models require dataset-specific training, leading to corresponding delays.
We introduce RAPID, a model that capitalizes on the inherent features of log data to enable anomaly detection without training delays.
arXiv Detail & Related papers (2023-11-09T06:11:44Z) - Log-based Anomaly Detection based on EVT Theory with feedback [31.949892354842525]
We present an accurate, lightweight, and adaptive log-based anomaly detection framework, referred to as SeaLog.
Our method introduces a Trie-based Detection Agent (TDA) that employs a lightweight, dynamically-growing trie structure for real-time anomaly detection.
To enhance TDA's accuracy in response to evolving log data, we enable it to receive feedback from experts.
arXiv Detail & Related papers (2023-06-08T08:34:58Z) - PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows.
Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
arXiv Detail & Related papers (2023-01-25T16:34:43Z) - Borrowing from Similar Code: A Deep Learning NLP-Based Approach for Log
Statement Automation [0.0]
We introduce an updated and improved log-aware code-clone detection method to predict the location of logging statements.
We incorporate natural language processing (NLP) and deep learning methods to automate the log statements' description prediction.
Our analysis shows that our hybrid NLP and code-clone detection approach (NLP CC'd) outperforms conventional clone detectors in finding log statement locations.
arXiv Detail & Related papers (2021-12-02T14:03:49Z) - LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak
Supervision [63.08516384181491]
We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts.
Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect.
Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
arXiv Detail & Related papers (2021-11-02T15:16:08Z) - A2Log: Attentive Augmented Log Anomaly Detection [53.06341151551106]
Anomaly detection becomes increasingly important for the dependability and serviceability of IT services.
Existing unsupervised methods need anomaly examples to obtain a suitable decision boundary.
We develop A2Log, which is an unsupervised anomaly detection method consisting of two steps: Anomaly scoring and anomaly decision.
arXiv Detail & Related papers (2021-09-20T13:40:21Z) - Experience Report: Deep Learning-based System Log Analysis for Anomaly
Detection [30.52620190783608]
We provide a review and evaluation on five popular models used by six state-of-the-art anomaly detectors.
Four of the selected methods are unsupervised and the remaining two are supervised.
We believe our work can serve as a basis in this field and contribute to the future academic researches and industrial applications.
arXiv Detail & Related papers (2021-07-13T08:10:47Z) - Information Theoretic Meta Learning with Gaussian Processes [74.54485310507336]
We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck.
By making use of variational approximations to the mutual information, we derive a general and tractable framework for meta learning.
arXiv Detail & Related papers (2020-09-07T16:47:30Z) - Process Discovery for Structured Program Synthesis [70.29027202357385]
A core task in process mining is process discovery which aims to learn an accurate process model from event log data.
In this paper, we propose to use (block-) structured programs directly as target process models.
We develop a novel bottom-up agglomerative approach to the discovery of such structured program process models.
arXiv Detail & Related papers (2020-08-13T10:33:10Z) - Self-Supervised Log Parsing [59.04636530383049]
Large-scale software systems generate massive volumes of semi-structured log records.
Existing approaches rely on log-specifics or manual rule extraction.
We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
arXiv Detail & Related papers (2020-03-17T19:25:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.