Data Quality Rule Generation with LLMs

• URL: http://arxiv.org/abs/2609.06053v1
• Date: Sat, 05 Sep 2026 12:17:31 GMT
• Title: Data Quality Rule Generation with LLMs
• Abstract summary: We introduce LeDQeR, an LLM-based DQ rule generation approach.<n>A large language model (LLM) generates candidates rules from an observed dirty data.<n>LeDQeR is able to produce effective and compact rule sets for various datasets and error types.
• Score: 1.3505192986379397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The validation of data, such as customer and employee data, is an important task in many organizations. Errors in data can have severe consequences. For example, a wrong drug unit in a patient record can lead to life-threatening medication errors, and a missing street number in an address to failed deliveries. Companies often employ rule-based enterprise data quality (DQ) tools, which allow domain experts to specify rules to validate the data over time. While rule-based DQ tools are computationally efficient and provide explainable reports, maintaining a comprehensive rule set manually is challenging, as domain experts often overlook essential rules, especially in complex domains and large data volumes. Hence, closing these gaps remains an open problem in practice. In this paper, we address the challenge of automated DQ rule generation. For this, we formalize a generalizable generate-filter framework and introduce LeDQeR, an LLM-based DQ rule generation approach. First, a large language model (LLM) generates candidates rules from an observed dirty data tuple for a given rule-based DQ tool syntax. Second, we apply four filter techniques that ensure the (i) executability, (ii) correctness, and (iii) generalizability, and avoid (iv) redundancy of the generated rules. An extensive experimental evaluation suggests that LeDQeR is able to produce effective and compact rule sets for various datasets and error types.

Related papers

• Taking a Closer Look at Warnings Generated by PMD and SonarQube, their Rules and Compliance to Established Coding Standards [2.6699776214769666]
  Static code analysis tools play a vital role in software development.<n>High false-positive and false-negative rates are reported for the best tools in the community.
  arXiv  Detail & Related papers  (2026-02-28T21:50:23Z)
• From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detection [89.52759572485276]
  ARC is a few-shot generalist GAD method that leverages in-context learning and requires only a few labeled normal samples at inference time.<n> ARC and ARC_zero effectively detect anomalies, exhibit strong generalization ability, and perform efficiently under few-shot and zero-shot settings.
  arXiv  Detail & Related papers  (2026-02-21T10:59:00Z)
• RulePilot: An LLM-Powered Agent for Security Rule Generation [15.638534363355342]
  Rule-based detection often identifies malicious logs based on the predefined grammar logic.<n>In this paper, we propose RulePilot, which mimics human expertise via LLM-based agent.<n>Using RulePilot, the security analysts do not need to write down the rules following the grammar, instead, they can just provide the annotations.
  arXiv  Detail & Related papers  (2025-11-15T13:59:16Z)
• Hey, That's My Data! Label-Only Dataset Inference in Large Language Models [63.35066172530291]
  CatShift is a label-only dataset-inference framework.<n>It capitalizes on catastrophic forgetting: the tendency of an LLM to overwrite previously learned knowledge when exposed to new data.
  arXiv  Detail & Related papers  (2025-06-06T13:02:59Z)
• Learning Rules from KGs Guided by Language Models [48.858741745144044]
  Rule learning methods can be applied to predict potentially missing facts. Ranking of rules is especially challenging over highly incomplete or biased KGs. With the recent rise of Language Models (LMs) several works have claimed that LMs can be used as alternative means for KG completion.
  arXiv  Detail & Related papers  (2024-09-12T09:27:36Z)
• RNR: Teaching Large Language Models to Follow Roles and Rules [153.6596303205894]
  We propose model, an automated data generation pipeline that generates diverse roles and rules from existing IFT instructions. This data can then be used to train models that follow complex system prompts. Our framework significantly improves role and rule following capability in large language models.
  arXiv  Detail & Related papers  (2024-09-10T06:07:32Z)
• RuleR: Improving LLM Controllability by Rule-based Data Recycling [28.74786215922553]
  Rule-based Data Recycling (RuleR) is a data augmentation method incorporating multiple constraints into the original data samples according to predefined rules.<n>Instead of creating new data from scratch, RuleR "recycles" existing data by simply applying rule-based edits to their responses and appending the rule-instructions in their original instructions.<n> Experimental results demonstrate RuleR's effectiveness in improving LLM controllability while maintaining general instruction-following capabilities.
  arXiv  Detail & Related papers  (2024-06-22T20:57:12Z)
• Automatic Question-Answer Generation for Long-Tail Knowledge [65.11554185687258]
  We propose an automatic approach to generate specialized QA datasets for tail entities. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets.
  arXiv  Detail & Related papers  (2024-03-03T03:06:31Z)
• FAIR: Filtering of Automatically Induced Rules [29.777290150010504]
  We propose an algorithm to filter rules from a large number of automatically induced rules. We show that our algorithm achieves statistically significant results in comparison to existing rule-filtering approaches.
  arXiv  Detail & Related papers  (2024-02-23T18:04:54Z)
• TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data [73.29220562541204]
  We consider harnessing the amazing power of language models (LLMs) to solve our task. We develop a TAT-LLM language model by fine-tuning LLaMA 2 with the training data generated automatically from existing expert-annotated datasets.
  arXiv  Detail & Related papers  (2024-01-24T04:28:50Z)
• LPRules: Rule Induction in Knowledge Graphs Using Linear Programming [4.94950858749529]
  Rule-based methods learn first-order logic rules that capture existing facts in an input graph and then use these rules for reasoning about missing facts. A major drawback of such methods is the lack of scalability to large datasets. We present a simple linear programming (LP) model to choose rules from a list of candidate rules and assign weights to them.
  arXiv  Detail & Related papers  (2021-10-15T17:58:16Z)
• Towards Learning Instantiated Logical Rules from Knowledge Graphs [20.251630903853016]
  We present GPFL, a probabilistic learner rule optimized to mine instantiated first-order logic rules from knowledge graphs. GPFL utilizes a novel two-stage rule generation mechanism that first generalizes extracted paths into templates that are acyclic abstract rules. We reveal the presence of overfitting rules, their impact on the predictive performance, and the effectiveness of a simple validation method filtering out overfitting rules.
  arXiv  Detail & Related papers  (2020-03-13T00:32:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.