SchemaCoder: Automatic Log Schema Extraction Coder with Residual Q-Tree Boosting

• URL: http://arxiv.org/abs/2508.18554v1
• Date: Mon, 25 Aug 2025 23:12:45 GMT
• Title: SchemaCoder: Automatic Log Schema Extraction Coder with Residual Q-Tree Boosting
• Abstract summary: We introduce a fully automated schema extraction framework, Coder, for a wide range of log file formats.<n>At its core, Coder features a novel Residual-Tree (Q-Tree) Boosting mechanism that iteratively refines schema extraction.<n>Our method partitions logs into semantic chunks via context-bounded segmentation, selects representative patterns using embedding-based sampling, and generates schema code through hierarchical Q-Tree-driven queries.
• Score: 14.204656295848357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Log schema extraction is the process of deriving human-readable templates from massive volumes of log data, which is essential yet notoriously labor-intensive. Recent studies have attempted to streamline this task by leveraging Large Language Models (LLMs) for automated schema extraction. However, existing methods invariably rely on predefined regular expressions, necessitating human domain expertise and severely limiting productivity gains. To fundamentally address this limitation, we introduce SchemaCoder, the first fully automated schema extraction framework applicable to a wide range of log file formats without requiring human customization within the flow. At its core, SchemaCoder features a novel Residual Question-Tree (Q-Tree) Boosting mechanism that iteratively refines schema extraction through targeted, adaptive queries driven by LLMs. Particularly, our method partitions logs into semantic chunks via context-bounded segmentation, selects representative patterns using embedding-based sampling, and generates schema code through hierarchical Q-Tree-driven LLM queries, iteratively refined by our textual-residual evolutionary optimizer and residual boosting. Experimental validation demonstrates SchemaCoder's superiority on the widely-used LogHub-2.0 benchmark, achieving an average improvement of 21.3% over state-of-the-arts.

Related papers

• LogNLQ: Natural-Language Log Querying with Parser-Induced and Semantically Grounded Schemas [40.92526942288794]
  We present LogNLQ, a framework that formulates natural-language querying over log-induced and semantically grounded schemas.<n>LogNLQ parses raw logs into template-ed relational tables, then applies dual-granularity semantic grounding to annotate both templates and parameter columns.<n>At query time, relevant schema candidates are retrieved via semantic search, and a large language model generates executablesql.
  arXiv  Detail & Related papers  (2026-07-04T14:02:06Z)
• SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text [23.835119696596095]
  SCOPE ( Construction and Ontology-induction Pipeline Evaluation) is a train-text-only benchmark for corpus-to-schema induction and optional schema fusion from raw text.<n>SCION ( Construction and Ontology-induction Pipeline Evaluation) is an auditable reference pipeline rather than a new extraction architecture.
  arXiv  Detail & Related papers  (2026-05-20T04:43:02Z)
• Stacked from One: Multi-Scale Self-Injection for Context Window Extension [69.24689919827817]
  modelname is a novel framework based on multi-grained context compression and query-aware information acquisition.<n>modelnameachieves performance superior or comparable to strong baselines.
  arXiv  Detail & Related papers  (2026-03-05T03:16:16Z)
• Performance Evaluation of LLMs in Automated RDF Knowledge Graph Generation [0.0]
  Large Language Models (LLMs) offer a promising approach to automate RDF knowledge graph generation.<n>We evaluate multiple LLMs and prompting strategies for automated RDF extraction using a controlled framework.<n>Our analysis shows that Few-Shot learning is the most effective strategy, with Llama achieving a 99.35% F1 score and 100% valid RDF output.
  arXiv  Detail & Related papers  (2026-02-06T06:30:35Z)
• Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version) [49.462399222747024]
  We propose a novel framework for the logical specification of non-Markovian rewards in Decision Processes (MDPs) with large state spaces.<n>Our approach leverages Linear Temporal Logic Modulo Theories over finite traces (LTLfMT)<n>We introduce a method based on reward machines and Hindsight Experience Replay (HER) to translate first-order logic specifications and address reward sparsity.
  arXiv  Detail & Related papers  (2026-02-05T22:11:28Z)
• Hierarchical Quantized Diffusion Based Tree Generation Method for Hierarchical Representation and Lineage Analysis [49.00783841494125]
  HDTree captures tree relationships within a hierarchical latent space using a unified hierarchical codebook and quantized diffusion processes.<n> HDTree's effectiveness is demonstrated through comparisons on both general-purpose and single-cell datasets.<n>These contributions provide a new tool for hierarchical lineage analysis, enabling more accurate and efficient modeling of cellular differentiation paths.
  arXiv  Detail & Related papers  (2025-06-29T15:19:13Z)
• SchemaGraphSQL: Efficient Schema Linking with Pathfinding Graph Algorithms for Text-to-SQL on Large-Scale Databases [1.6544167074080365]
  We present a zero-shot, training-free schema linking approach that first constructs a schema graph based on foreign key relations.<n>We apply classical path-finding algorithms and post-processing to identify the optimal sequence of tables and columns that should be joined.<n>Our method achieves state-of-the-art results on the BIRD benchmark, outperforming previous specialized, fine-tuned, and complex multi-step LLM-based approaches.
  arXiv  Detail & Related papers  (2025-05-23T20:42:36Z)
• SchemaAgent: A Multi-Agents Framework for Generating Relational Database Schema [35.57815867567431]
  Existing efforts are mostly based on customized rules or conventional deep learning models, often producing relational schema.<n>We propose a unified LLM-based multi-agent framework for the automated generation of high-quality database schema.Agent.<n>We incorporate dedicated roles for reflection and inspection, alongside an innovative error detection and correction mechanism to identify rectify issues across various phases.
  arXiv  Detail & Related papers  (2025-03-31T09:39:19Z)
• RASD: Retrieval-Augmented Speculative Decoding [5.3926068062773895]
  Speculative decoding accelerates inference in large language models (LLMs)<n>This paper proposes RASD (Retrieval-Augmented Speculative Decoding), which adopts retrieval methods to enhance model-based speculative decoding.
  arXiv  Detail & Related papers  (2025-03-05T12:10:14Z)
• Effective Instruction Parsing Plugin for Complex Logical Query Answering on Knowledge Graphs [51.33342412699939]
  Knowledge Graph Query Embedding (KGQE) aims to embed First-Order Logic (FOL) queries in a low-dimensional KG space for complex reasoning over incomplete KGs. Recent studies integrate various external information (such as entity types and relation context) to better capture the logical semantics of FOL queries. We propose an effective Query Instruction Parsing (QIPP) that captures latent query patterns from code-like query instructions.
  arXiv  Detail & Related papers  (2024-10-27T03:18:52Z)
• Latent Logic Tree Extraction for Event Sequence Explanation from LLMs [19.90330712436838]
  Modern high-stakes systems, such as healthcare or robotics, often generate vast streaming event sequences. Our goal is to design an efficient, plug-and-play tool to elicit logic tree-based explanations from Large Language Models (LLMs) to provide customized insights into each observed event sequence. In the online setting, our locally built, lightweight model will iteratively extract the most relevant rules from LLMs for each sequence using only a few iterations.
  arXiv  Detail & Related papers  (2024-06-03T09:10:42Z)
• Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction [12.455647753787442]
  We propose a three-phase framework named Extract-Define-Canonicalize (EDC) EDC is flexible in that it can be applied to settings where a pre-defined target schema is available and when it is not. We demonstrate EDC is able to extract high-quality triplets without any parameter tuning and with significantly larger schemas compared to prior works.
  arXiv  Detail & Related papers  (2024-04-05T02:53:51Z)
• Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging [18.823038918091207]
  We introduce a cutting-edge texttextbfLog parsing framework with textbfEntropy sampling and chain-of-thought textbfMerging (model)<n>To discard the tedious manual rules, we propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs.<n>We have conducted experiments on large-scale public datasets.
  arXiv  Detail & Related papers  (2024-02-28T09:51:55Z)
• SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization [51.67317895094664]
  This paper studies file-level code summarization, which can assist programmers in understanding and maintaining large source code projects. We propose SparseCoder, an identifier-aware sparse transformer for effectively handling long code sequences.
  arXiv  Detail & Related papers  (2024-01-26T09:23:27Z)
• Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner [56.08919422452905]
  We propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR) Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. We outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.
  arXiv  Detail & Related papers  (2022-05-18T21:52:11Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.