The Minimum Description Length Principle for Pattern Mining: A Survey

• URL: http://arxiv.org/abs/2007.14009v5
• Date: Thu, 28 Jul 2022 06:25:49 GMT
• Title: The Minimum Description Length Principle for Pattern Mining: A Survey
• Abstract summary: This describes the Minimum Description Length principle applied to pattern mining. After giving an outline of relevant concepts from information theory and coding, we review MDL-based methods for mining various types of data and patterns.
• Score: 0.07614628596146598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This is about the Minimum Description Length (MDL) principle applied to pattern mining. The length of this description is kept to the minimum. Mining patterns is a core task in data analysis and, beyond issues of efficient enumeration, the selection of patterns constitutes a major challenge. The MDL principle, a model selection method grounded in information theory, has been applied to pattern mining with the aim to obtain compact high-quality sets of patterns. After giving an outline of relevant concepts from information theory and coding, as well as of work on the theory behind the MDL and similar principles, we review MDL-based methods for mining various types of data and patterns. Finally, we open a discussion on some issues regarding these methods, and highlight currently active related data analysis problems.

Related papers

• On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks [56.98385132295952]
  We evaluate how well chain-of-thought approaches generalize on a simple planning task.<n>We find that reasoning traces which combine multiple text formats yield the best (and non-trivial) OOD generalization.<n> purely text-based models consistently outperform those utilizing image-based inputs.
  arXiv  Detail & Related papers  (2026-02-17T09:51:40Z)
• Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces [2.0789230137053014]
  Test-time scaling has enabled a new class of Large Language Models (LLMs) that are able to reason through complex problems.<n>We compare the performance of medium-sized LLMs on Math problems after post-training on two kinds of reasoning traces.
  arXiv  Detail & Related papers  (2025-11-24T17:26:58Z)
• SoK: Data Minimization in Machine Learning [49.60064304454055]
  Data minimization (DM) describes the principle of collecting only the data strictly necessary for a given task.<n>The relevance of data minimization is particularly pronounced in machine learning (ML) applications.<n>Existing work on other ML privacy and security topics often addresses concerns relevant to DMML without explicitly acknowledging the connection.<n>This work introduces a comprehensive framework for DMML, including a unified data pipeline, adversaries, and points of minimization.
  arXiv  Detail & Related papers  (2025-08-14T17:00:13Z)
• A MIND for Reasoning: Meta-learning for In-context Deduction [3.4383794581359184]
  We propose Meta-learning for In-context Deduction (MIND), a novel few-shot meta-learning fine-tuning approach.<n>Our results show that MIND significantly improves generalization in small LMs ranging from 1.5B to 7B parameters.<n>Remarkably, small models fine-tuned with MIND outperform state-of-the-art LLMs, such as GPT-4o and o3-mini, on this task.
  arXiv  Detail & Related papers  (2025-05-20T13:00:48Z)
• Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs [51.09983600916971]
  Recent research indicates that models demonstrating linearity enhance the performance of task arithmetic. We argue that this linearity already exists within the model's submodules. We propose an innovative model merging strategy that independently merges these submodules.
  arXiv  Detail & Related papers  (2025-04-15T06:23:24Z)
• Minimum Description Length of a Spectrum Variational Autoencoder: A Theory [1.3735277588793995]
  We argue that Minimum Description Length (MDL) is essential to deep learning. We introduce a novel theoretical framework for designing and evaluating deep Variational Autoencoders (VAEs) based on MDL. We claim that a Spectrum VAE understands the data distribution in the most appropriate way when the MDL is achieved.
  arXiv  Detail & Related papers  (2025-04-01T03:37:18Z)
• Understanding Chain-of-Thought in LLMs through Information Theory [16.78730663293352]
  We formalize Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) through an information-theoretic lens. Specifically, our framework quantifies the information gain' at each reasoning step, enabling the identification of failure modes. We demonstrate the efficacy of our approach through extensive experiments on toy and GSM-8K data, where it significantly outperforms existing outcome-based methods.
  arXiv  Detail & Related papers  (2024-11-18T19:14:36Z)
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time [51.5039731721706]
  MindStar is a purely inference-based searching method for large language models. It formulates reasoning tasks as searching problems and proposes two search ideas to identify the optimal reasoning paths. It significantly enhances the reasoning abilities of open-source models, such as Llama-2-13B and Mistral-7B, and achieves comparable performance to GPT-3.5 and Grok-1.
  arXiv  Detail & Related papers  (2024-05-25T15:07:33Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Granger Causal Inference in Multivariate Hawkes Processes by Minimum Message Length [0.0]
  We propose an optimization criterion and model selection algorithm based on the minimum message length (MML) principle. While most of the state-of-art methods using lasso-type penalization tend to overfitting in scenarios with short time horizons, the proposed MML-based method achieves high F1 scores in these settings.
  arXiv  Detail & Related papers  (2023-09-05T08:13:34Z)
• MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning [53.90744622542961]
  Reasoning in mathematical domains remains a significant challenge for small language models (LMs) We introduce a new method that exploits existing mathematical problem datasets with diverse annotation styles. Experimental results show that our strategy enables a LLaMA-7B model to outperform prior approaches.
  arXiv  Detail & Related papers  (2023-07-16T05:41:53Z)
• On Interpretable Approaches to Cluster, Classify and Represent Multi-Subspace Data via Minimum Lossy Coding Length based on Rate-Distortion Theory [0.0]
  Clustering, classify and represent are three fundamental objectives of learning from high-dimensional data with intrinsic structure. This paper introduces three interpretable approaches, i.e., segmentation (clustering) via the Minimum Lossy Coding Length criterion, classification via the Minimum Incremental Coding Length criterion and representation via the Maximal Coding Rate Reduction criterion.
  arXiv  Detail & Related papers  (2023-02-21T01:15:08Z)
• Contrast Pattern Mining: A Survey [54.06874773607785]
  It is difficult for new researchers in the field to understand the general situation of the field in a short period of time. First, we present an in-depth understanding of CPM, including basic concepts, types, mining strategies, and metrics for assessing discriminative ability. We classify CPM methods according to their characteristics into boundary-based algorithms, tree-based algorithms, evolutionary fuzzy system-based algorithms, decision tree-based algorithms, and other algorithms.
  arXiv  Detail & Related papers  (2022-09-27T17:11:12Z)
• The whole and the parts: the MDL principle and the a-contrario framework [4.970364068620608]
  This work explores the connections between the Minimum Description Length (MDL) principle and the a-contrario framework for structure detection proposed by Desolneux, Moisan and Morel. The MDL principle focuses on the best interpretation for the whole data while the a-contrario approach concentrates on detecting parts of the data with anomalous statistics.
  arXiv  Detail & Related papers  (2021-12-13T18:07:26Z)
• Gaussian Determinantal Processes: a new model for directionality in data [10.591948377239921]
  In this work, we investigate a parametric family of Gaussian DPPs with a clearly interpretable effect of parametric modulation on the observed points. We show that parameter modulation impacts the observed points by introducing directionality in their repulsion structure, and the principal directions correspond to the directions of maximal dependency. This model readily yields a novel and viable alternative to Principal Component Analysis (PCA) as a dimension reduction tool that favors directions along which the data is most spread out.
  arXiv  Detail & Related papers  (2021-11-19T00:57:33Z)
• Rissanen Data Analysis: Examining Dataset Characteristics via Description Length [78.42578316883271]
  We introduce a method to determine if a certain capability helps to achieve an accurate model of given data. Since minimum program length is uncomputable, we estimate the labels' minimum description length (MDL) as a proxy. We call the method Rissanen Data Analysis (RDA) after the father of MDL.
  arXiv  Detail & Related papers  (2021-03-05T18:58:32Z)
• Meta Cyclical Annealing Schedule: A Simple Approach to Avoiding Meta-Amortization Error [50.83356836818667]
  We develop a novel meta-regularization objective using it cyclical annealing schedule and it maximum mean discrepancy (MMD) criterion. The experimental results show that our approach substantially outperforms standard meta-learning algorithms.
  arXiv  Detail & Related papers  (2020-03-04T04:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.