JTON: A Token-Efficient JSON Superset with Zen Grid Tabular Encoding for Large Language Models

• URL: http://arxiv.org/abs/2604.05865v1
• Date: Tue, 07 Apr 2026 13:26:23 GMT
• Title: JTON: A Token-Efficient JSON Superset with Zen Grid Tabular Encoding for Large Language Models
• Abstract summary: Zen Grid factors column headers into a single row and encodes values with domains.<n>A Rust/PyO3 reference implementation adds SIMD-accelerated syntactic parsing at 1.4x the speed of Python's.<n>Generation tests on 12 LLMs yield 100% validity in both few-shot and zero-shot settings.
• Score: 0.010480658740597679
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When LLMs process structured data, the serialization format directly affects cost and context utilization. Standard JSON wastes tokens repeating key names in every row of a tabular array--overhead that scales linearly with row count. This paper presents JTON (JSON Tabular Object Notation), a strict JSON superset whose main idea, Zen Grid, factors column headers into a single row and encodes values with semicolons, preserving JSON's type system while cutting redundancy. Across seven real-world domains, Zen Grid reduces token counts by 15-60% versus JSON compact (28.5% average; 32% with bare_strings). Comprehension tests on 10 LLMs show a net +0.3 pp accuracy gain over JSON: four models improve, three hold steady, and three dip slightly. Generation tests on 12 LLMs yield 100% syntactic validity in both few-shot and zero-shot settings. A Rust/PyO3 reference implementation adds SIMD-accelerated parsing at 1.4x the speed of Python's json module. Code, a 683-vector test suite, and all experimental data are publicly available.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• HoWToBench: Holistic Evaluation for LLM's Capability in Human-level Writing using Tree of Writing [84.84959282480641]
  We propose Tree-of-Writing (ToW) to resolve the implicit inconsistency often found when LLM-as-a-judge aggregates all sub-features in text evaluation.<n>We also present HowToBench, a large-scale Chinese writing benchmark encompassing 12 genres and 1302 instructions across three task categories.
  arXiv  Detail & Related papers  (2026-04-21T04:26:39Z)
• ONTO: A Token-Efficient Columnar Notation for LLM Input Optimization [0.0]
  ONTO is a columnar notation that declares field names once per entity and arranges values in pipe-delimited rows with indentation-based hierarchy.<n>Onto eliminates per-record key repetition while preserving human readability and nested structure support.
  arXiv  Detail & Related papers  (2026-04-19T16:05:33Z)
• Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation [0.0]
  Token-Oriented Object Notation (TOON) aims to replace as a serialization format for passing structured data to LLMs with significantly reduced token usage.<n>To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation and comparing plain generation vs structured output.
  arXiv  Detail & Related papers  (2026-02-08T11:58:03Z)
• JSON Whisperer: Efficient JSON Editing with LLMs [1.0535472555708638]
  Large language models (LLMs) can modify documents through natural language commands, but current approaches regenerate entire structures for each edit, resulting in computational inefficiency.<n>We present Whisperer, a framework that enables LLMs to generate RFC 6902 diff patches-expressing only the necessary modifications-rather than complete documents.
  arXiv  Detail & Related papers  (2025-10-06T11:36:46Z)
• A Multi-Language Object-Oriented Programming Benchmark for Large Language Models [61.267115598083315]
  A survey of 35 existing benchmarks uncovers three major imbalances.<n>85.7% focus on a single programming language.<n>94.3% target only function-level or statement-level tasks.<n>Over 80% include fewer than ten test cases on average.
  arXiv  Detail & Related papers  (2025-09-30T11:30:08Z)
• DeepJSONEval: Benchmarking Complex Nested JSON Data Mining for Large Language Models [6.653834890554154]
  Multi-layer nested structures organize data into key-value pairs, arrays, and nested objects.<n>For instance, in news aggregation, a object can nest an article's metadata (title, author, date), content (text, multimedia), and multimedia information (multimedia, caption) hierarchically.<n>We introduce DeepJSONEval, a novel benchmark featuring 2100 multi-domain instances with deep nested structures, categorized by difficulty.
  arXiv  Detail & Related papers  (2025-09-30T08:18:20Z)
• Learning to Generate Structured Output with Schema Reinforcement Learning [83.09230124049667]
  This study investigates the structured generation capabilities of large language models (LLMs)<n>We find that the latest LLMs are still struggling to generate a valid string.<n>Our models demonstrate significant improvement in both generating outputs and downstream tasks.
  arXiv  Detail & Related papers  (2025-02-26T06:45:29Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding [84.04706075621013]
  We present a general-purpose, modular neural semantic parsing framework based on token-level fine-grained query understanding. Our framework consists of three modules: named entity recognizer (NER), neural entity linker (NEL) and neural entity linker (NSP)
  arXiv  Detail & Related papers  (2022-09-28T21:00:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.