Autoregressive Synthesis of Sparse and Semi-Structured Mixed-Type Data

• URL: http://arxiv.org/abs/2603.01444v1
• Date: Mon, 02 Mar 2026 04:47:50 GMT
• Title: Autoregressive Synthesis of Sparse and Semi-Structured Mixed-Type Data
• Abstract summary: We present Origami, an autoregressive transformer-based architecture that tokenizes data records into sequences of key, value and structural tokens.<n>This representation handles sparsity, mixed types and hierarchical structure without flattening or imputation.
• Score: 2.191505742658975
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Synthetic data generation is a critical capability for data sharing, privacy compliance, system benchmarking and test data provisioning. Existing methods assume dense, fixed-schema tabular data, yet this assumption is increasingly at odds with modern data systems - from document databases, REST APIs to data lakes - which store and exchange data in sparse, semi-structured formats like JSON. Applying existing tabular methods to such data requires flattening of nested data into wide, sparse tables which scales poorly. We present Origami, an autoregressive transformer-based architecture that tokenizes data records, including nested objects and variable length arrays, into sequences of key, value and structural tokens. This representation natively handles sparsity, mixed types and hierarchical structure without flattening or imputation. Origami outperforms baselines spanning GAN, VAE, diffusion and autoregressive architectures on fidelity, utility and detection metrics across nearly all settings, while maintaining high privacy scores. On semi-structured datasets with up to 38% sparsity, baseline synthesizers either fail to scale or degrade substantially, while Origami maintains high-fidelity synthesis that is harder to distinguish from real data. To the best of our knowledge, Origami is the first architecture capable of natively modeling and generating semi-structured data end-to-end.

Related papers

• DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams [48.434767119224375]
  We propose a paradigm shift towards Agentic Data Tailoring.<n>To overcome the data scarcity bottleneck in training such high-order capabilities, we design a two-stage pipeline.<n>$textDataClaw_0$-9B model synergizes Supervised Fine-Tuning with Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2026-06-19T11:31:43Z)
• Generative Data Transformation: From Mixed to Unified Data [57.84692191369066]
  textscTaesar is a emphdata-centric framework for textbftarget-textbfal textbfregeneration.<n>It encodes cross-domain context into target sequences, enabling standard models to learn intricate dependencies without complex fusion architectures.
  arXiv  Detail & Related papers  (2026-02-26T08:30:09Z)
• StructSynth: Leveraging LLMs for Structure-Aware Tabular Data Synthesis in Low-Data Regimes [15.476662936746989]
  Struct Synth is a novel framework that integrates the generative power of Large Language Models with robust structural control.<n>It produces synthetic data with significantly higher structural integrity and downstream utility than state-of-the-art methods.<n>It proves especially effective in challenging low-data scenarios, successfully navigating the trade-off between privacy preservation and statistical fidelity.
  arXiv  Detail & Related papers  (2025-08-04T16:55:02Z)
• RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models [83.6013616017646]
  RelDiff is a novel diffusion generative model that synthesizes complete relational databases by explicitly modeling their foreign key graph structure.<n>RelDiff consistently outperforms prior methods in producing realistic and coherent synthetic relational databases.
  arXiv  Detail & Related papers  (2025-05-31T21:01:02Z)
• LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion [49.898152180805454]
  Synthetic datasets must maintain domain-specific logical consistency.<n>Existing generative models often overlook these inter-column relationships.<n>This study presents the first method to effectively preserve inter-column relationships without requiring domain knowledge.
  arXiv  Detail & Related papers  (2025-03-04T00:47:52Z)
• ORIGAMI: A generative transformer architecture for predictions from semi-structured data [3.5639148953570836]
  ORIGAMI is a transformer-based architecture that processes nested key/value pairs.<n>By reformulating classification as next-token prediction, ORIGAMI naturally handles both single-label and multi-label tasks.
  arXiv  Detail & Related papers  (2024-12-23T07:21:17Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Framework [18.11940247961923]
  In this paper, we introduce high-order structural causal information as natural prior knowledge. We propose multiple benchmark tasks, high-order metrics, and causal inference tasks as downstream tasks for evaluating the quality of synthetic data.
  arXiv  Detail & Related papers  (2024-06-12T15:12:49Z)
• Generating Realistic Synthetic Relational Data through Graph Variational Autoencoders [47.89542334125886]
  We combine the variational autoencoder framework with graph neural networks to generate realistic synthetic relational databases. The results indicate that real databases' structures are accurately preserved in the resulting synthetic datasets.
  arXiv  Detail & Related papers  (2022-11-30T10:40:44Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Information [0.6443952406204634]
  We propose a novel approach to formulate, extract, encode and inject hierarchical structure information explicitly into an extractive summarization model. Using various experimental settings on three datasets (i.e., CNN/DailyMail, PubMed and arXiv), our HiStruct+ model outperforms a strong baseline collectively.
  arXiv  Detail & Related papers  (2022-03-17T21:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.