LogPrism: Unifying Structure and Variable Encoding for Effective Log Compression

• URL: http://arxiv.org/abs/2601.17482v1
• Date: Sat, 24 Jan 2026 15:12:31 GMT
• Title: LogPrism: Unifying Structure and Variable Encoding for Effective Log Compression
• Abstract summary: "parse-then-compress" paradigm limits effectiveness by treating log parsing and compression as isolated objectives.<n>We propose LogPrism, a framework that bridges the gap via unified redundancy encoding.<n> experiments on 16 benchmark datasets confirm that LogPrism establishes a new state-of-the-art.
• Score: 41.26407915252864
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The prevailing "parse-then-compress" paradigm in log compression fundamentally limits effectiveness by treating log parsing and compression as isolated objectives. While parsers prioritize semantic accuracy (i.e., event identification), they often obscure deep correlations between static templates and dynamic variables that are critical for storage efficiency. In this paper, we investigate this misalignment through a comprehensive empirical study and propose LogPrism, a framework that bridges the gap via unified redundancy encoding. Rather than relying on a rigid pre-parsing step, LogPrism dynamically integrates structural extraction with variable encoding by constructing a Unified Redundancy Tree (URT). This hierarchical approach effectively mines "structure+variable" co-occurrence patterns, capturing deep contextual redundancies while accelerating processing through pre-emptive pattern encoding. Extensive experiments on 16 benchmark datasets confirm that LogPrism establishes a new state-of-the-art. It achieves the highest compression ratio on 13 datasets, surpassing leading baselines by margins of 4.7% to 80.9%, while delivering superior throughput at 29.87 MB/s (1.68$\times$~43.04$\times$ faster than competitors). Moreover, when configured in single-archive mode to maximize global pattern discovery, LogPrism outperforms the best baseline by 19.39% in compression ratio while maintaining a 2.62$\times$ speed advantage.

Related papers

• Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models [6.94102129809152]
  Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation.<n>We study a minimalist compound sparsity framework that first applies low-rank approximation and channel pruning to obtain a statically compressed backbone.
  arXiv  Detail & Related papers  (2026-06-29T01:33:30Z)
• Planning-aligned Token Compression for Long-Context Autonomous Driving [95.59023657139208]
  We propose a planning-aligned working memory framework built on conditional VQ-VA.<n>Compression is conditioned on both historical trajectory and a learned planning intent.<n>We evaluate on high-signal dynamic scenarios where historical context is most critical for behavior correctness.
  arXiv  Detail & Related papers  (2026-06-05T17:16:21Z)
• CLAD: Efficient Log Anomaly Detection Directly on Compressed Representations [2.487363589269089]
  We introduce CLAD, the first deep learning framework to perform LAD directly on compressed byte streams.<n>We propose a purpose-built architecture integrating a dilated convolutional byte encoder, a hybrid Transformer--mLSTM, and four-way aggregation pooling.<n> evaluated across five datasets, CLAD achieves a state-of-the-art average F1-score of 0.9909 and outperforms the best baseline by 2.72 percentage points.
  arXiv  Detail & Related papers  (2026-04-14T17:57:01Z)
• Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression [10.689840449667974]
  Large Language Models are constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache.<n>Existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient.<n>We propose Swift-SVD, an activation-aware, closed-form compression framework that guarantees theoretical optimum, practical efficiency and numerical stability.
  arXiv  Detail & Related papers  (2026-04-02T04:40:50Z)
• LogFold: Compressing Logs with Structured Tokens and Hybrid Encoding [36.603581308724465]
  We introduce LogFold, a novel log compression method consisting of four components.<n>A token analyzer classifies tokens as structured, unstructured, or static types; a processor mines recurring patterns within structured tokens based on their skeletons; a hybrid encoder tailors data representation according to token types.<n>Experiments on 16 public log datasets demonstrate that LogFold surpasses state-of-the-art baselines, achieving average compression ratio improvements by 11.11%, with a compression speed of 9.842 MB/s.
  arXiv  Detail & Related papers  (2026-03-21T03:27:37Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• ROCKET: Rapid Optimization via Calibration-guided Knapsack Enhanced Truncation for Efficient Model Compression [5.280540253822294]
  ROCKET is a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines.<n>ROCKET consistently outperforms existing compression approaches across different model architectures at 20-50% compression rates.
  arXiv  Detail & Related papers  (2026-02-11T16:34:52Z)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression [55.63153956934198]
  Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs)<n>Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios.<n>We propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-02-09T06:57:15Z)
• DeLog: An Efficient Log Compression Framework with Pattern Signature Synthesis [30.690382753027137]
  A novel log compressor implements a Pattern Signature Synthesis mechanism to achieve efficient pattern-based grouping.<n>On 16 public and 10 production datasets, DeLog achieves state-of-the-art compression ratio and speed.
  arXiv  Detail & Related papers  (2026-01-21T15:26:09Z)
• SimpleMem: Efficient Lifelong Memory for LLM Agents [73.74399447715052]
  We introduce SimpleMem, an efficient memory framework based on semantic lossless compression.<n>We propose a three-stage pipeline designed to maximize information density and token utilization.<n> Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost.
  arXiv  Detail & Related papers  (2026-01-05T21:02:49Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization [5.857877898558651]
  Chain-of-Thought (CoT) reasoning enhances the problem-solving ability of large language models (LLMs) but leads to substantial inference overhead.<n>This paper investigates efficient CoT transfer across models of different scales and architectures through an adaptive reasoning summarization framework.
  arXiv  Detail & Related papers  (2025-11-07T22:35:31Z)
• DeepPrune: Parallel Scaling without Inter-trace Redundancy [53.62015294143274]
  Over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation.<n>We propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning.<n>Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient.
  arXiv  Detail & Related papers  (2025-10-09T17:24:54Z)
• OjaKV: Context-Aware Online Low-Rank KV Cache Compression with Oja's Rule [54.37983890753086]
  We introduce OjaKV, a framework that integrates a strategic hybrid storage policy with online subspace adaptation.<n>OjaKV preserves crucial first and most recent tokens in full-rank, maintaining high-fidelity anchors for attention.<n>It applies low-rank compression by incrementally adapting the projection basis using Oja's algorithm for online principal component analysis.
  arXiv  Detail & Related papers  (2025-09-25T21:42:27Z)
• Generative Latent Diffusion for Efficient Spatiotemporal Data Reduction [11.494915987840876]
  Experimental results across multiple datasets show that our method achieves up to 10 times higher compression ratios than rule-based state-of-the-art compressors such as SZ3, and up to 63 percent better performance than leading learning-based methods under the same reconstruction error.
  arXiv  Detail & Related papers  (2025-07-02T20:27:38Z)
• Efficient Token Compression for Vision Transformer with Spatial Information Preserved [59.79302182800274]
  Token compression is essential for reducing the computational and memory requirements of transformer models.<n>We propose an efficient and hardware-compatible token compression method called Prune and Merge.
  arXiv  Detail & Related papers  (2025-03-30T14:23:18Z)
• LeCo: Lightweight Compression via Learning Serial Correlations [9.108815508920882]
  Lightweight data compression is a key technique that allows column stores to exhibit superior performance for analytical queries. We propose LeCo (i.e., Learned Compression), a framework that uses machine learning to remove the serial redundancy in a value sequence automatically. We observe up to 5.2x speed up in a data analytical query in the Arrow columnar execution engine and a 16% increase in RocksDB's throughput.
  arXiv  Detail & Related papers  (2023-06-27T10:46:36Z)
• Sequential Encryption of Sparse Neural Networks Toward Optimum Representation of Irregular Sparsity [9.062897838978955]
  We study fixed-to-fixed encryption architecture/algorithm to support fine-grained pruning methods. We demonstrate that our proposed compression scheme achieves almost the maximum compression ratio for the Transformer and ResNet-50.
  arXiv  Detail & Related papers  (2021-05-05T05:14:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.