SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation

• URL: http://arxiv.org/abs/2604.24368v1
• Date: Mon, 27 Apr 2026 12:03:14 GMT
• Title: SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation
• Abstract summary: We introduce SAGE (Sparse Adaptive Guidance), a novel LLM-based generation framework that enforces sparse and dynamic dependency guidance.<n>We show that SAGE improves data fidelity and downstream utility, boosting F1 scores by 10% compared to previous LLM-based methods, but also reduces policy violations by one point.
• Score: 24.77840620410903
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generating high-fidelity synthetic tabular data remains a critical challenge for enhancing data availability in privacy-sensitive and low-resource domains. Recent approaches leverage LLMs by representing table rows as sequences, yet suffer from two fundamental limitations: (1) they model feature dependencies densely, introducing spurious correlations; and (2) they assume static relationships between features, ignoring how these dependencies vary with feature values. To overcome these limitations, we introduce SAGE (Sparse Adaptive Guidance), a novel LLM-based generation framework that enforces sparse and dynamic dependency guidance. SAGE discretizes features into value-aware pseudo-features and constructs a mutual information-based sparse dependency graph. This graph adaptively guides generation through explicit context selection or implicit logit correction, enabling LLMs to focus on truly relevant information during synthesis. Our extensive experiments across six datasets and multiple tasks reveal that SAGE not only improves data fidelity and downstream utility, boosting F1 scores by 10% compared to previous LLM-based methods, but also reduces policy violations by one point. These results highlight the importance of adaptive structure in tabular data generation and provide new insights into context-sensitive control of LLMs.

Related papers

• Sparse Personalized Text Generation with Multi-Trajectory Reasoning [95.19317487799292]
  PAT (Personalization with Aligned Trajectories) is a reasoning framework for cold-start LLM personalization.<n>It first retrieves information along two complementary trajectories: writing-style cues from stylistically similar users and topic-specific context from preference-aligned users.<n>It then employs a reinforcement learning-based, iterative dual-reasoning mechanism that enables the LLM to jointly refine and integrate these signals.
  arXiv  Detail & Related papers  (2026-04-27T20:58:49Z)
• Self-Reinforcing Controllable Synthesis of Rare Relational Data via Bayesian Calibration [12.065354583341518]
  In this work, we propose RDDG, which is a unified in-context learning framework that employs progressive chain-of-thought steps to generate data.<n>It incorporates a self-reinforcing feedback mechanism that provides automatic assessments of the quality of the generated data.<n> Experimental results on multiple real and synthetic datasets demonstrate that RDDG outperforms existing approaches in both data fidelity and downstream imbalanced classification performance.
  arXiv  Detail & Related papers  (2026-04-18T04:15:24Z)
• Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference [7.817259532290553]
  Large language models (LLMs) are increasingly used for text-rich graph machine learning tasks.<n>Despite a surge of interest, the field lacks a principled understanding of the capabilities of LLMs in their interaction with graph data.
  arXiv  Detail & Related papers  (2025-09-23T00:46:21Z)
• Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimes [7.036974567001374]
  ReFine is a framework that guides generation toward domain-specific feature distribution.<n>Experiments on various regression and classification benchmarks demonstrate that ReFine consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-09-12T04:34:46Z)
• Doubling Your Data in Minutes: Ultra-fast Tabular Data Generation via LLM-Induced Dependency Graphs [15.476573983202162]
  Tabular data is critical across diverse domains, yet high-quality datasets remain scarce due to privacy concerns and the cost of collection.<n>We propose SPADA for SPArse Dependency-driven Augmentation, a lightweight generative framework that captures sparse dependencies via an LLM-induced graph.<n> Experiments on four datasets show that SPADA reduces constraint violations by 4% compared to diffusion-based methods and accelerates generation by nearly 9,500 times over LLM-based baselines.
  arXiv  Detail & Related papers  (2025-07-25T14:43:50Z)
• Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models [24.77840620410903]
  We propose GraDe (Graph-Guided Dependency Learning), a novel method that integrates sparse dependency graphs into Large Language Models' attention mechanism.<n>GraDe employs a lightweight dynamic graph learning module guided by externally extracted functional dependencies, prioritizing key feature interactions while suppressing irrelevant ones.<n>Our experiments across diverse real-world datasets demonstrate that GraDe outperforms existing LLM-based approaches by up to 12% on complex datasets.
  arXiv  Detail & Related papers  (2025-07-24T15:22:27Z)
• LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion [49.898152180805454]
  Synthetic datasets must maintain domain-specific logical consistency.<n>Existing generative models often overlook these inter-column relationships.<n>This study presents the first method to effectively preserve inter-column relationships without requiring domain knowledge.
  arXiv  Detail & Related papers  (2025-03-04T00:47:52Z)
• Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models [60.00178316095646]
  Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using datasets like NLI.<n>Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency.<n>We propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence.<n> Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis.
  arXiv  Detail & Related papers  (2025-02-19T12:07:53Z)
• DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph [70.79413606968814]
  We introduce Dynamic Evaluation of LLMs via Adaptive Reasoning Graph Evolvement (DARG) to dynamically extend current benchmarks with controlled complexity and diversity. Specifically, we first extract the reasoning graphs of data points in current benchmarks and then perturb the reasoning graphs to generate novel testing data. Such newly generated test samples can have different levels of complexity while maintaining linguistic diversity similar to the original benchmarks.
  arXiv  Detail & Related papers  (2024-06-25T04:27:53Z)
• Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes [57.62036621319563]
  We introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. We demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators.
  arXiv  Detail & Related papers  (2023-12-19T12:34:46Z)
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
  We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
  arXiv  Detail & Related papers  (2023-08-23T09:45:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.