DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks

• URL: http://arxiv.org/abs/2604.20553v1
• Date: Wed, 22 Apr 2026 13:37:13 GMT
• Title: DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks
• Abstract summary: Machine-learning-based generalizations like Drain are fast but accuracy often degrades on complex variables.<n>This paper presents DeepParse, a hybrid framework that automatically mines reusable variable patterns from small log samples.<n>By separating the reasoning phase from execution, DeepParse enables accurate, scalable, and cost-efficient log structuring.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern distributed systems produce massive, heterogeneous logs essential for reliability, security, and anomaly detection. Converting these free-form messages into structured templates (log parsing) is challenging due to evolving formats and limited labeled data. Machine-learning-based parsers like Drain are fast but accuracy often degrades on complex variables, while Large Language Models (LLMs) offer better generalization but incur prohibitive inference costs. This paper presents DeepParse, a hybrid framework that automatically mines reusable variable patterns from small log samples using an LLM, then applies them deterministically through the Drain algorithm. By separating the reasoning phase from execution, DeepParse enables accurate, scalable, and cost-efficient log structuring without relying on brittle handcrafted rules or per-line neural inference. Across 16 benchmark datasets, DeepParse achieves higher accuracy in variable extraction (97.6% average Parsing Accuracy) and better consistency than both heuristic and LLM-only baselines. Integrating DeepParse into an anomaly detection pipeline reduced false alarms by over 30% and reduced inference latency by 36% compared to heuristic baselines.

Related papers

• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Parser-Free Querying of Security Logs [2.9970870516340504]
  Security analysts routinely query system logs to detect threats and investigate incidents.<n>Each log source uses its own semi-structured format: logs are cheap to produce, but expensive to use.<n>We present Sieve, a system that generates executable query code from natural-language security questions.
  arXiv  Detail & Related papers  (2026-05-21T05:58:04Z)
• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching [66.39914384073145]
  We propose a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates.<n>We find that step-level recombination is most beneficial on harder problems.<n>Our training-free framework improves average accuracy by up to 2 across six math and coding tasks.
  arXiv  Detail & Related papers  (2026-02-26T11:08:39Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding [60.407727995313074]
  We introduce adaptive parallel decoding (APD), a novel method that dynamically adjusts the number of tokens sampled in parallel.<n>APD provides markedly higher throughput with minimal quality degradations on downstream benchmarks.
  arXiv  Detail & Related papers  (2025-05-31T06:10:10Z)
• LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models [19.657278472819588]
  We introduce Log-LLM, a novel log integrated with LLM capabilities. We address the intricate challenge of parsing granularity, proposing a new metric to allow users to calibrate granularity to their specific needs. Our method's efficacy is empirically demonstrated through evaluations on the Loghub-2k and the large-scale LogPub benchmark.
  arXiv  Detail & Related papers  (2024-08-25T05:34:24Z)
• HELP: Hierarchical Embeddings-based Log Parsing [0.25112747242081457]
  Logs are a first-hand source of information for software maintenance and failure diagnosis. Log parsing is a prerequisite for automated log analysis tasks such as anomaly detection, troubleshooting, and root cause analysis. Existing online parsing algorithms are susceptible to log drift, where slight log changes create false positives that drown out real anomalies.
  arXiv  Detail & Related papers  (2024-08-15T17:54:31Z)
• LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Models [3.7960472831772774]
  This paper introduces LibreLog, an unsupervised log parsing approach that enhances privacy and reduces operational costs while achieving state-of-the-art parsing accuracy. Our evaluation on LogHub-2.0 shows that LibreLog achieves 25% higher parsing accuracy and processes 2.7 times faster compared to state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-08-02T21:54:13Z)
• Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction [15.93927602769091]
  Recent emergence of large language models (LLMs) has demonstrated strong abilities in understanding natural language and code.<n>Ada is an effective and adaptive log parsing framework using LLMs with self-generated in-context learning (SG-ICL) and self-correction.<n>Ada outperforms state-of-the-art methods across all metrics, even in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-06-05T15:31:43Z)
• Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging [18.823038918091207]
  We introduce a cutting-edge texttextbfLog parsing framework with textbfEntropy sampling and chain-of-thought textbfMerging (model)<n>To discard the tedious manual rules, we propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs.<n>We have conducted experiments on large-scale public datasets.
  arXiv  Detail & Related papers  (2024-02-28T09:51:55Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.