D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring

• URL: http://arxiv.org/abs/2608.28236v1
• Date: Fri, 28 Aug 2026 11:53:16 GMT
• Title: D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring
• Abstract summary: Three conditions cause existing methods to degrade, namely data scarcity, high process entropy and distributional shift.<n>This paper introduces D-TAIA, a framework for a joint next activity and remaining time prediction task via parameter-efficient fine-tuning of an FM backbone.
• Score: 0.30586855806896046
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Predictive Process Monitoring (PPM) enables organizations to forecast future process behavior, such as the next activity and remaining time of ongoing cases. In practice, three conditions cause existing methods to degrade, namely data scarcity, high process entropy and distributional shift. While Foundation Models (FMs), especially Large Language Models (LLMs), offer a new paradigm through broad sequential reasoning, adapting them to multi-task PPM under these conditions remains an open challenge. Existing FM-based approaches either lack mechanisms for handling distributional shift or rely on direct regression heads that can be structurally misaligned with continuous time prediction tasks. This paper introduces D-TAIA (Domain-aware Training and Attention-based Inference Architecture), a framework for a joint next activity and remaining time prediction task via parameter-efficient fine-tuning of an FM backbone. Our approach combines domain-aware triplet loss (DATL) pre-training with FAISS-based nearest neighbor retrieval for remaining time prediction, and adopts the TAIA inference strategy to preserve pre-trained sequential reasoning during fine-tuning. Evaluated across four real-world event logs, D-TAIA consistently shows SOTA or competitive performance compared to a fine-tuned LLM and a recurrent neural network baseline. Ablation studies confirm that techniques from NLP and computer vision can be transferred effectively to PPM with only a 10M-parameter backbone, though component contributions vary by dataset entropy.

Related papers

• Efficient Online Continual Foundation Model Fine-Tuning for Predictive Process Monitoring [0.34376560669160394]
  We propose the first framework for continual fine-tuning of FMs for Predictive Process Monitoring.<n>It adapts loss-plateau drift detection to autonomously identify task boundaries in event streams.<n>We evaluate our approach on nine event streams covering synthetic and real-world concept drift scenarios.
  arXiv  Detail & Related papers  (2026-08-28T11:53:26Z)
• LeapTS: Rethinking Time Series Forecasting as Adaptive Multi-Horizon Scheduling [74.94985663101906]
  We present LeapTS, a novel framework that reformulates time series forecasting as a dynamic scheduling process over the prediction horizon.<n>We show that LeapTS improves overall forecasting performance by at least 7.4% while achieving a 2.6$times$ to 5.3$times$ inference speedup over representative Transformer-based models.
  arXiv  Detail & Related papers  (2026-05-11T09:54:02Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• Time Series Foundation Models for Process Model Forecasting [8.339024524110828]
  Process Model Forecasting aims to predict how the control-flow structure of a process evolves over time.<n>Machine learning and deep learning models provide only modest gains over statistical baselines.<n>We investigate Time Series Foundation Models (TSFMs) as an alternative for PMF.
  arXiv  Detail & Related papers  (2025-12-08T15:08:50Z)
• In-Context Learning of Temporal Point Processes with Foundation Inference Models [12.037629169675938]
  Current approaches to MTPP inference rely on training separate, specialized models for each target system.<n>We pretrain a deep neural network to infer, in-context, the conditional intensity functions of event histories from a context defined by sets of event sequences.<n>Our amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-29T13:28:06Z)
• On the Simplification of Neural Network Architectures for Predictive Process Monitoring [0.5735035463793009]
  We analyze how reducing model complexity, both in terms of parameter count and architectural depth, impacts predictive performance.<n>We show that shrinking the Transformer model by 85% results in only a 2-3% drop in performance.<n>Overall, our findings suggest that substantial model simplification can preserve predictive accuracy.
  arXiv  Detail & Related papers  (2025-09-21T16:21:45Z)
• Domain Adaptation of LLMs for Process Data [7.611051482274626]
  Large Language Models (LLMs) have emerged as a prominent area of interest across various research domains, including Process Mining (PM)<n>This study investigates the direct adaptation of pretrained LLMs to process data without natural language reformulation.<n>More specifically, we focus on parameter-efficient fine-tuning techniques to mitigate the computational overhead typically associated with such models.
  arXiv  Detail & Related papers  (2025-09-03T09:21:35Z)
• WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training [64.0932926819307]
  We present Warmup-Stable and Merge (WSM), a framework that establishes a formal connection between learning rate decay and model merging.<n>WSM provides a unified theoretical foundation for emulating various decay strategies.<n>Our framework consistently outperforms the widely-adopted Warmup-Stable-Decay (WSD) approach across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-07-23T16:02:06Z)
• DIDS: Domain Impact-aware Data Sampling for Large Language Model Training [61.10643823069603]
  We present Domain Impact-aware Data Sampling (DIDS) for large language models.<n>DIDS group training data based on learning effects, where a proxy language model and dimensionality reduction are employed.<n>It achieves 3.4% higher average performance while maintaining comparable training efficiency.
  arXiv  Detail & Related papers  (2025-04-17T13:09:38Z)
• Cumulative Distribution Function based General Temporal Point Processes [49.758080415846884]
  CuFun model represents a novel approach to TPPs that revolves around the Cumulative Distribution Function (CDF) Our approach addresses several critical issues inherent in traditional TPP modeling. Our contributions encompass the introduction of a pioneering CDF-based TPP model, the development of a methodology for incorporating past event information into future event prediction.
  arXiv  Detail & Related papers  (2024-02-01T07:21:30Z)
• Transformer Hawkes Process [79.16290557505211]
  We propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies. THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.
  arXiv  Detail & Related papers  (2020-02-21T13:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.