An Innovative Next Activity Prediction Approach Using Process Entropy and DAW-Transformer

• URL: http://arxiv.org/abs/2502.10573v1
• Date: Fri, 14 Feb 2025 22:02:00 GMT
• Title: An Innovative Next Activity Prediction Approach Using Process Entropy and DAW-Transformer
• Authors: Hadi Zare, Mostafa Abbasi, Maryam Ahang, Homayoun Najjaran,
• Abstract summary: This paper proposes an entropy-driven model selection approach and DAW-Transformer to integrate all attributes with a dynamic window for better accuracy.<n>Experiments were conducted on six public datasets, and the performance was evaluated with process entropy.
• Score: 7.139735602624267
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Purpose - In Business Process Management (BPM), accurate prediction of the next activities is vital for operational efficiency and decision-making. Current Artificial Intelligence (AI)/Machine Learning (ML) models struggle with the complexity and evolving nature of business process event logs, balancing accuracy and interpretability. This paper proposes an entropy-driven model selection approach and DAW-Transformer, which stands for Dynamic Attribute-Aware Transformer, to integrate all attributes with a dynamic window for better accuracy. Design/methodology/approach - This paper introduces a novel next-activity prediction approach that uses process entropy to assess the complexity of event logs and dynamically select the most suitable ML model. A new transformer-based architecture with multi-head attention and dynamic windowing mechanism, DAW-Transformer, is proposed to capture long-range dependencies and utilize all relevant event log attributes. Experiments were conducted on six public datasets, and the performance was evaluated with process entropy. Finding - The results demonstrate the effectiveness of the approach across these publicly available datasets. DAW-Transformer achieved superior performance, especially on high-entropy datasets such as Sepsis exceeding Limited window Multi-Transformers by 4.69% and a benchmark CNN-LSTM-SAtt model by 3.07%. For low-entropy datasets like Road Traffic Fine, simpler, more interpretable algorithms like Random Forest performed nearly as well as the more complex DAW-Transformer and offered better handling of imbalanced data and improved explainability. Originality/ value - This work's novelty lies in the proposed DAW-Transformer, with a dynamic window and considering all relevant attributes. Also, entropy-driven selection methods offer a robust, accurate, and interpretable solution for next-activity prediction.

Related papers

• DDP-WM: Disentangled Dynamics Prediction for Efficient World Models [79.53092337527382]
  We introduce DDP-WM, a novel world model centered on the principle of Disentangled Dynamics Prediction.<n>DDP-WM realizes this decomposition through an architecture that integrates efficient historical processing with dynamic localization.<n>Experiments demonstrate that DDP-WM achieves significant efficiency and performance across diverse tasks.
  arXiv  Detail & Related papers  (2026-02-02T08:04:25Z)
• EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience [44.734653745434834]
  We introduce EvoCUA, a native computer use agentic model.<n>Unlike static imitation, EvoCUA integrates data generation and policy optimization into a self-sustaining evolutionary cycle.<n>EvoCUA significantly outperforms the previous best open-source model, OpenCUA-72B.
  arXiv  Detail & Related papers  (2026-01-22T11:36:43Z)
• Comprehensive Attribute Encoding and Dynamic LSTM HyperModels for Outcome Oriented Predictive Business Process Monitoring [5.634923879819779]
  Predictive Business Process Monitoring aims to forecast future outcomes of ongoing business processes.<n>Existing methods often lack flexibility to handle real-world challenges such as simultaneous events, class imbalance, and multi-level attributes.<n>We propose a suite of dynamic LSTM HyperModels that integrate two-level hierarchical encoding for event and sequence attributes.<n> specialized LSTM variants for simultaneous event modeling, leveraging multidimensional embeddings and time-difference flag augmentation.
  arXiv  Detail & Related papers  (2025-06-04T08:27:58Z)
• TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer [27.798471160707436]
  We propose TIDFormer, a dynamic graph TransFormer that fully exploits Temporal and Interactive Dynamics.<n>To model the temporal and interactive dynamics, respectively, we utilize the calendar-based time partitioning information.<n>In addition, we jointly model temporal and interactive features by capturing potential changes in historical interaction patterns.
  arXiv  Detail & Related papers  (2025-05-31T07:23:05Z)
• Context-Aware Rule Mining Using a Dynamic Transformer-Based Framework [8.52080590054588]
  This study proposes a dynamic rule data mining algorithm based on an improved Transformer architecture. We show that the improved Transformer model has achieved significant improvements in rule mining accuracy, coverage, and stability. Future research will focus on optimizing computational efficiency and combining more deep learning technologies to expand the application scope of the algorithm.
  arXiv  Detail & Related papers  (2025-03-14T06:37:04Z)
• Multi-Agent Sampling: Scaling Inference Compute for Data Synthesis with Tree Search-Based Agentic Collaboration [81.45763823762682]
  This work aims to bridge the gap by investigating the problem of data synthesis through multi-agent sampling.<n>We introduce Tree Search-based Orchestrated Agents(TOA), where the workflow evolves iteratively during the sequential sampling process.<n>Our experiments on alignment, machine translation, and mathematical reasoning demonstrate that multi-agent sampling significantly outperforms single-agent sampling as inference compute scales.
  arXiv  Detail & Related papers  (2024-12-22T15:16:44Z)
• PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulation [68.17081518640934]
  We propose a PrIrmitive-driVen waypOinT-aware world model for Robotic manipulation (PIVOT-R) PIVOT-R consists of a Waypoint-aware World Model (WAWM) and a lightweight action prediction module. Our PIVOT-R outperforms state-of-the-art open-source models on the SeaWave benchmark, achieving an average relative improvement of 19.45% across four levels of instruction tasks.
  arXiv  Detail & Related papers  (2024-10-14T11:30:18Z)
• Sampling Foundational Transformer: A Theoretical Perspective [12.7600763629179]
  We propose Foundational Sampling Transformer (SFT) that can work on multiple data modalities. SFT has achieved competitive results on many benchmarks, while being faster in inference, compared to other very specialized models.
  arXiv  Detail & Related papers  (2024-08-11T16:53:09Z)
• Dynamic Model Switching for Improved Accuracy in Machine Learning [0.0]
  We introduce an adaptive ensemble that intuitively transitions between CatBoost and XGBoost. The user sets a benchmark, say 80% accuracy, prompting the system to dynamically shift to the new model only if it guarantees improved performance. This dynamic model-switching mechanism aligns with the evolving nature of data in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-31T00:13:02Z)
• AutoFT: Learning an Objective for Robust Fine-Tuning [60.641186718253735]
  Foundation models encode rich representations that can be adapted to downstream tasks by fine-tuning. Current approaches to robust fine-tuning use hand-crafted regularization techniques. We propose AutoFT, a data-driven approach for robust fine-tuning.
  arXiv  Detail & Related papers  (2024-01-18T18:58:49Z)
• A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case [4.0466311968093365]
  This work proposed an encoder-transformer architecture-based framework for time series prediction. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset. To enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed.
  arXiv  Detail & Related papers  (2023-08-19T02:30:35Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Optimizing Non-Autoregressive Transformers with Contrastive Learning [74.46714706658517]
  Non-autoregressive Transformers (NATs) reduce the inference latency of Autoregressive Transformers (ATs) by predicting words all at once rather than in sequential order. In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
  arXiv  Detail & Related papers  (2023-05-23T04:20:13Z)
• Full Stack Optimization of Transformer Inference: a Survey [58.55475772110702]
  Transformer models achieve superior accuracy across a wide range of applications. The amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate. There has been an increased focus on making Transformer models more efficient.
  arXiv  Detail & Related papers  (2023-02-27T18:18:13Z)
• Universal Transformer Hawkes Process with Adaptive Recursive Iteration [4.624987488467739]
  Asynchronous events sequences are widely distributed in the natural world and human activities, such as earthquakes records, users activities in social media and so on. How to distill the information from these seemingly disorganized data is a persistent topic that researchers focus on. The one of the most useful model is the point process model, and on the basis, the researchers obtain many noticeable results. In recent years, point process models on the foundation of neural networks, especially recurrent neural networks (RNN) are proposed and compare with the traditional models, their performance are greatly improved.
  arXiv  Detail & Related papers  (2021-12-29T09:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.