Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2505.22041v1
• Date: Wed, 28 May 2025 07:03:46 GMT
• Title: Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation
• Abstract summary: We propose an approach that requires neither a dedicated process model nor resource-intensive fine-tuning to detect undesired process behavior.<n>Instead, we use Retrieval Augmented Generation (RAG) to provide an LLM with direct access to a knowledge base.<n>Our evaluation shows that our approach outperforms fine-tuned LLMs in detecting undesired behavior.
• Score: 0.5461938536945723
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conformance checking techniques detect undesired process behavior by comparing process executions that are recorded in event logs to desired behavior that is captured in a dedicated process model. If such models are not available, conformance checking techniques are not applicable, but organizations might still be interested in detecting undesired behavior in their processes. To enable this, existing approaches use Large Language Models (LLMs), assuming that they can learn to distinguish desired from undesired behavior through fine-tuning. However, fine-tuning is highly resource-intensive and the fine-tuned LLMs often do not generalize well. To address these limitations, we propose an approach that requires neither a dedicated process model nor resource-intensive fine-tuning to detect undesired process behavior. Instead, we use Retrieval Augmented Generation (RAG) to provide an LLM with direct access to a knowledge base that contains both desired and undesired process behavior from other processes, assuming that the LLM can transfer this knowledge to the process at hand. Our evaluation shows that our approach outperforms fine-tuned LLMs in detecting undesired behavior, demonstrating that RAG is a viable alternative to resource-intensive fine-tuning, particularly when enriched with relevant context from the event log, such as frequent traces and activities.

Related papers

• Monitoring Data-aware Temporal Properties (Extended Version) [56.386411908764494]
  We consider anticipatory monitoring of linear-time properties enriched with an arbitrary SMT theory over finite traces.<n>Anticipatory monitoring in this setting is highly challenging, as the monitoring state depends on both the trace prefix seen so far and all its possible finite continuations.<n>We present and formally prove the correctness of a novel foundational framework for monitoring properties in an expressive fragment offMT.
  arXiv  Detail & Related papers  (2026-05-14T10:23:11Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• ProRAG: Process-Supervised Reinforcement Learning for Retrieval-Augmented Generation [54.071574153853994]
  ProRAG is a process-supervised reinforcement learning framework designed to integrate learned step-level supervision into the online optimization loop.<n>Our framework consists of four stages: (1) Supervised Policy Warmup to initialize the model with a structured reasoning format; (2) construction of an MCTS-based Process Reward Model (PRM) to quantify intermediate reasoning quality; (3) PRM-Guided Reasoning Refinement to align the policy with fine-grained process preferences; and (4) Process-Supervised Reinforcement Learning with a dual-granularity advantage mechanism.
  arXiv  Detail & Related papers  (2026-01-29T16:04:59Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• Generation of Programmatic Rules for Document Forgery Detection Using Large Language Models [10.32461766065764]
  Document forgery poses a growing threat to legal, economic, and governmental processes.<n>Existing plausibility checks are manually implemented by software engineers, which is time-consuming.<n>Recent advances in code generation with large language models (LLMs) offer new potential for automating and scaling the generation of these checks.
  arXiv  Detail & Related papers  (2025-12-22T10:08:25Z)
• LLMs that Understand Processes: Instruction-tuning for Semantics-Aware Process Mining [1.3050391810942312]
  semantics-aware process mining focuses on what behavior should be possible in a process.<n>Large Language Models (LLMs) provide a powerful means for tackling semantics-aware tasks.<n>This paper investigates the potential of instruction-tuning for semantics-aware process mining.
  arXiv  Detail & Related papers  (2025-08-22T10:13:13Z)
• Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inference [78.08901120841833]
  We propose a method to detect the knowledge boundary of Visual Large Language Models (VLLMs)<n>We show that our method successfully depicts a VLLM's knowledge boundary based on which we are able to reduce indiscriminate retrieval while maintaining or improving the performance.
  arXiv  Detail & Related papers  (2025-02-25T09:32:08Z)
• Stream-Based Active Learning for Process Monitoring [1.2221087476416055]
  This article presents a novel stream-based active learning strategy for Statistical process monitoring (SPM) The ultimate goal is to optimize labeling resources constrained by a limited budget and dynamically update the possible OC states.
  arXiv  Detail & Related papers  (2024-11-19T15:27:54Z)
• Tractable Offline Learning of Regular Decision Processes [50.11277112628193]
  This work studies offline Reinforcement Learning (RL) in a class of non-Markovian environments called Regular Decision Processes (RDPs) Ins, the unknown dependency of future observations and rewards from the past interactions can be captured experimentally. Many algorithms first reconstruct this unknown dependency using automata learning techniques.
  arXiv  Detail & Related papers  (2024-09-04T14:26:58Z)
• Mining Constraints from Reference Process Models for Detecting Best-Practice Violations in Event Log [1.389948527681755]
  We propose a framework for mining declarative best-practice constraints from a reference model collection. We demonstrate the capability of our framework to detect best-practice violations through an evaluation based on real-world process model collections and event logs.
  arXiv  Detail & Related papers  (2024-07-02T15:05:37Z)
• Adaptive Retention & Correction: Test-Time Training for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task.<n>We name our approach Adaptive Retention & Correction (ARC)<n>ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• Using Reinforcement Learning to Optimize Responses in Care Processes: A Case Study on Aggression Incidents [0.8158530638728501]
  We train a Markov decision process using event data from a care process. The goal was to find optimal policies for staff members when clients are displaying any type of aggressive behavior. Results showed that the policies derived from these algorithms are similar to the most frequent actions currently used.
  arXiv  Detail & Related papers  (2023-10-02T08:43:29Z)
• CAPE: Corrective Actions from Precondition Errors using Large Language Models [8.547766794082184]
  We propose a novel approach that attempts to propose corrective actions to resolve precondition errors during planning. CAPE improves the quality of generated plans by leveraging few-shot reasoning from action preconditions. Our improvements transfer to a Boston Dynamics Spot robot with a set of skills (specified in language) and associated preconditions, where CAPE improves the correctness metric of the executed task plans by 76.49% compared to SayCan.
  arXiv  Detail & Related papers  (2022-11-17T23:14:51Z)
• Detecting Surprising Situations in Event Data [0.45119235878273]
  In existing work, it is usually assumed that the set of problematic process instances in which an undesirable outcome occurs is known prior or is easily detectable. We formulate finding the process enhancement areas as a context-sensitive anomaly/outlier detection problem. We aim to characterize those situations where process performance/outcome is significantly different from what was expected.
  arXiv  Detail & Related papers  (2022-08-29T11:33:58Z)
• A Monitoring and Discovery Approach for Declarative Processes Based on Streams [0.0]
  We present a discovery algorithm that extracts declarative processes as Dynamic Condition Response (DCR) graphs from event streams. Streams are monitored to generate temporal representations of the process, later processed to generate declarative models.
  arXiv  Detail & Related papers  (2022-08-10T14:25:35Z)
• Chain of Thought Imitation with Procedure Cloning [129.62135987416164]
  We propose procedure cloning, which applies supervised sequence prediction to imitate the series of expert computations. We show that imitating the intermediate computations of an expert's behavior enables procedure cloning to learn policies exhibiting significant generalization to unseen environment configurations.
  arXiv  Detail & Related papers  (2022-05-22T13:14:09Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Certified Reinforcement Learning with Logic Guidance [78.2286146954051]
  We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs) The algorithm is guaranteed to synthesise a control policy whose traces satisfy the specification with maximal probability.
  arXiv  Detail & Related papers  (2019-02-02T20:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.