Source-Grounded Data Generation for Text-to-JSON Learning

• URL: http://arxiv.org/abs/2606.20072v1
• Date: Thu, 18 Jun 2026 10:47:22 GMT
• Title: Source-Grounded Data Generation for Text-to-JSON Learning
• Abstract summary: We propose STAGE, a source-grounded text-to-JSON data generation pipeline.<n>We show that STAGE produces stronger training data than existing approaches.<n>This improves Q3-4B exact match from 31.37% to 74.27% and value accuracy from 851.6% to 90.69%.
• Score: 24.82878938113544
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: From financial filings to clinical records, legacy industries rely heavily on long, unstructured documents to store high-value information. Reliably extracting this information into structured, machine-readable representations is a key prerequisite to making the contents accessible to automated systems. JSON is a natural target for such structured extraction, yet constructing reliable and scalable text-to-JSON training data remains challenging. To address this gap, we propose STAGE (Spreadsheet-grounded Text-to-JSON Artifact GEneration), a source-grounded data generation pipeline that constructs reports and JSON schema by using LLMs for scalable synthesis while validating ground-truth values against the underlying spreadsheet. Evaluations on STAGE-Eval, our source-grounded benchmark with an 851-example test set, show that STAGE produces stronger training data than existing approaches. This improves Qwen3-4B exact match from 31.37% to 74.27% and value accuracy from 45.46% to 90.69%.

Related papers

• The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output Quality in Large Language Models [0.23332469289621785]
  We introduce SOB (The Structured Output Benchmark), a multi-source benchmark spanning three source modalities.<n>All models receive a text-normalized representation of their context regardless of source modality.<n>Our results reveal a consistent pattern: models achieve near-perfect schema compliance, yet the best Value Accuracy, measured by exact leaf-value match, reaches only 83.0% on text.
  arXiv  Detail & Related papers  (2026-04-28T08:27:01Z)
• Autoregressive Synthesis of Sparse and Semi-Structured Mixed-Type Data [2.191505742658975]
  We present Origami, an autoregressive transformer-based architecture that tokenizes data records into sequences of key, value and structural tokens.<n>This representation handles sparsity, mixed types and hierarchical structure without flattening or imputation.
  arXiv  Detail & Related papers  (2026-03-02T04:47:50Z)
• DeepJSONEval: Benchmarking Complex Nested JSON Data Mining for Large Language Models [6.653834890554154]
  Multi-layer nested structures organize data into key-value pairs, arrays, and nested objects.<n>For instance, in news aggregation, a object can nest an article's metadata (title, author, date), content (text, multimedia), and multimedia information (multimedia, caption) hierarchically.<n>We introduce DeepJSONEval, a novel benchmark featuring 2100 multi-domain instances with deep nested structures, categorized by difficulty.
  arXiv  Detail & Related papers  (2025-09-30T08:18:20Z)
• StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation [8.251302684712773]
  StructText is an end-to-end framework for automatically generating high-fidelity benchmarks for key-value extraction from text.<n>We evaluate the proposed method on 71,539 examples across 49 documents.
  arXiv  Detail & Related papers  (2025-07-28T21:20:44Z)
• Learning to Generate Structured Output with Schema Reinforcement Learning [83.09230124049667]
  This study investigates the structured generation capabilities of large language models (LLMs)<n>We find that the latest LLMs are still struggling to generate a valid string.<n>Our models demonstrate significant improvement in both generating outputs and downstream tasks.
  arXiv  Detail & Related papers  (2025-02-26T06:45:29Z)
• Discourse Features Enhance Detection of Document-Level Machine-Generated Content [53.41994768824785]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>Existing MGC detectors often focus solely on surface-level information, overlooking implicit and structural features.<n>We introduce novel methodologies and datasets to overcome these challenges.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text [29.03935605732864]
  We introduce StrucText-Eval, a benchmark to evaluate how well large language models understand and reason through structured text. We show that while open-source LLMs achieve a maximum accuracy of 74.9% on the standard dataset, their performance drops significantly to 45.8% on the harder dataset. In contrast, human participants reach an accuracy of 92.6% on StrucText-Eval-Hard, highlighting LLMs' current limitations in handling intricate structural information.
  arXiv  Detail & Related papers  (2024-06-15T12:48:00Z)
• Improving Text Embeddings with Large Language Models [59.930513259982725]
  We introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data.
  arXiv  Detail & Related papers  (2023-12-31T02:13:18Z)
• TarGEN: Targeted Data Generation with Large Language Models [51.87504111286201]
  TarGEN is a multi-step prompting strategy for generating high-quality synthetic datasets. We augment TarGEN with a method known as self-correction empowering LLMs to rectify inaccurately labeled instances. A comprehensive analysis of the synthetic dataset compared to the original dataset reveals similar or higher levels of dataset complexity and diversity.
  arXiv  Detail & Related papers  (2023-10-27T03:32:17Z)
• Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation [92.1582872870226]
  We propose a new grounded keys-to-text generation task. The task is to generate a factual description about an entity given a set of guiding keys, and grounding passages. Inspired by recent QA-based evaluation measures, we propose an automatic metric, MAFE, for factual correctness of generated descriptions.
  arXiv  Detail & Related papers  (2022-12-04T23:59:41Z)
• Partially-Aligned Data-to-Text Generation with Distant Supervision [69.15410325679635]
  We propose a new generation task called Partially-Aligned Data-to-Text Generation (PADTG) It is more practical since it utilizes automatically annotated data for training and thus considerably expands the application domains. Our framework outperforms all baseline models as well as verify the feasibility of utilizing partially-aligned data.
  arXiv  Detail & Related papers  (2020-10-03T03:18:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.