David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM

• URL: http://arxiv.org/abs/2606.15868v1
• Date: Sun, 14 Jun 2026 15:40:21 GMT
• Title: David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM
• Abstract summary: Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM)<n>No benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP.<n>Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.

Related papers

• MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation [51.15583711678273]
  We propose a novel closed-loop system that bridges data generation, model training, and evaluation.<n>Within each loop iteration, the MLLM-DataEngine first analyzes the weakness of the model based on the evaluation results.<n>We find MLLM-DataEngine could boost the MLLMs capability in a targeted and automatic manner without human participants.
  arXiv  Detail & Related papers  (2026-07-07T08:35:38Z)
• NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints [100.02131897927484]
  This paper focuses on the native training of Multimodal Large Language Models (MLLMs) in an end-to-end manner.<n>We propose a native MLLM called NaViL, combined with a simple and cost-effective recipe.<n> Experimental results on 14 multimodal benchmarks confirm the competitive performance of NaViL against existing MLLMs.
  arXiv  Detail & Related papers  (2025-10-09T17:59:37Z)
• xLSTM Scaling Laws: Competitive Performance with Linear Time-Complexity [22.40851170527]
  Scaling laws play a central role in the success of Large Language Models.<n>Recent alternatives such as xLSTM offer linear complexity with respect to context length.<n>xLSTM's advantage widens as training and inference contexts grow.
  arXiv  Detail & Related papers  (2025-10-02T17:14:34Z)
• Fusing Large Language Models with Temporal Transformers for Time Series Forecasting [17.549938378193282]
  Large language models (LLMs) have demonstrated powerful capabilities in performing various tasks.<n>LLMs are proficient at reasoning over discrete tokens and semantic patterns.<n> vanilla Transformers often struggle to learn high-level semantic patterns.
  arXiv  Detail & Related papers  (2025-07-14T09:33:40Z)
• Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignment [16.147350486106777]
  Large Language Models (LLMs) have demonstrated impressive performance in time series analysis.<n>However, since LLMs are not designed for time series tasks, simpler models like linear regressions can often achieve comparable performance.
  arXiv  Detail & Related papers  (2024-10-16T07:47:31Z)
• PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning [17.73193523921637]
  Large language models (LLMs) have shown remarkable abilities in diverse natural language processing (NLP) tasks. LLMs generally undergo supervised fine-tuning (SFT) followed by preference alignment to be usable in downstream applications. This paper introduces PAFT, a new PArallel training paradigm for effective LLM Fine-Tuning.
  arXiv  Detail & Related papers  (2024-06-25T20:11:37Z)
• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models [52.98743860365194]
  We propose a new fine-tuning method called Self-Play fIne-tuNing (SPIN) At the heart of SPIN lies a self-play mechanism, where the LLM refines its capability by playing against instances of itself. This sheds light on the promise of self-play, enabling the achievement of human-level performance in LLMs without the need for expert opponents.
  arXiv  Detail & Related papers  (2024-01-02T18:53:13Z)
• LLM-augmented Preference Learning from Natural Language [19.700169351688768]
  Large Language Models (LLMs) are equipped to deal with larger context lengths. LLMs can consistently outperform the SotA when the target text is large. Few-shot learning yields better performance than zero-shot learning.
  arXiv  Detail & Related papers  (2023-10-12T17:17:27Z)
• MLLM-DataEngine: An Iterative Refinement Approach for MLLM [62.30753425449056]
  We propose a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results. For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data. For quality, we resort to GPT-4 to generate high-quality data with each given data type.
  arXiv  Detail & Related papers  (2023-08-25T01:41:04Z)
• LLM-Pruner: On the Structural Pruning of Large Language Models [65.02607075556742]
  Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. We tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures.
  arXiv  Detail & Related papers  (2023-05-19T12:10:53Z)
• Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations [51.77000472945441]
  Long Short-Term Memory (LSTM) and Transformers are two popular neural architectures used for natural language processing tasks. In practice, it is often observed that Transformer models have better representation power than LSTM. We study such practical differences between LSTM and Transformer and propose an explanation based on their latent space decomposition patterns.
  arXiv  Detail & Related papers  (2021-12-16T19:56:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.