Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation

• URL: http://arxiv.org/abs/2603.03306v1
• Date: Sun, 08 Feb 2026 11:58:03 GMT
• Title: Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation
• Abstract summary: Token-Oriented Object Notation (TOON) aims to replace as a serialization format for passing structured data to LLMs with significantly reduced token usage.<n>To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation and comparing plain generation vs structured output.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently presented Token-Oriented Object Notation (TOON) aims to replace JSON as a serialization format for passing structured data to LLMs with significantly reduced token usage. While showing solid accuracy in LLM comprehension, there is a lack of tests against JSON generation. Though never present in training data, TOON syntax is simple enough to suggest one-shot in-context learning could support accurate generation. The inevitable prompt overhead can be an acceptable trade-off for shorter completions. To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation pipeline, and comparing plain JSON generation vs structured output (via constrained decoding) JSON generation vs TOON one-shot in-context learning generation. JSON structured output was included to establish a minimum token budget baseline and to set a starting point for future experiments testing TOON constrained decoding inference enforcement. Key findings: TOON shows promising accuracy/token consumption ratio for in-domain generation tasks, though this advantage is often reduced by the "prompt tax" of instructional overhead in shorter contexts. Plain JSON generation shows the best one-shot and final accuracy, even compared with constrained decoding structured output, where the only significant advantage is the lowest token usage as a trade-off for slightly decreased accuracy overall and significant degradation for some models. Notably, for simple structures, this "lowest token usage" of constrained decoding outperformed even TOON, hinting that TOON enforcing via frameworks such as xgrammar may not yield the desired results. Furthermore, the results suggest a scaling hypothesis: TOON's true efficiency potential likely follows a non-linear curve, shining only beyond a specific point where cumulative syntax savings amortize the initial prompt overhead.

Related papers

• Stay Within Your Bounds: Distance-Guided Decoding for Guaranteed Context-Free Grammar Compliance [18.38520405734125]
  We propose a lookahead-guided decoding framework for context-free grammars based on pushdown automata.<n>The resulting decoder is syntactically sound: every output is accepted by the target grammar.
  arXiv  Detail & Related papers  (2026-08-28T11:44:53Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems [0.0]
  Large language models in Agentic AI systems consume tool schemas and execution results as structured data.<n>Recent work proposes token-optimized alternatives such as TOON (Token-Oriented Object Notation) and TRON (Token Reduced Object Notation) as more compact replacements.<n>We evaluate TOON and TRON on four agentic benchmarks (BFCL, MCPBenchPP, MCP-Universe, StableTool) and five open-weight LLMs.
  arXiv  Detail & Related papers  (2026-05-28T09:38:13Z)
• TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints [0.7734726150561086]
  TruncProof is a novel grammar-constrained generation method for machine-readable outputs.<n>We show that TruncProof can be effectively combined with advanced decoding strategies, resulting in grammatically valid and semantically accurate outputs.
  arXiv  Detail & Related papers  (2026-05-13T06:49:08Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs [57.217593337454026]
  TokenSqueeze is a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data.<n>We show that TokenSqueeze reduces token usage while maintaining accuracy on the MATH500 benchmark.
  arXiv  Detail & Related papers  (2025-11-17T10:38:56Z)
• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution [3.4666771782038652]
  Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency.<n>We introduce FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens.<n>We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning.
  arXiv  Detail & Related papers  (2025-10-18T10:22:13Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• Learning to Generate Structured Output with Schema Reinforcement Learning [83.09230124049667]
  This study investigates the structured generation capabilities of large language models (LLMs)<n>We find that the latest LLMs are still struggling to generate a valid string.<n>Our models demonstrate significant improvement in both generating outputs and downstream tasks.
  arXiv  Detail & Related papers  (2025-02-26T06:45:29Z)
• Predicting Through Generation: Why Generation Is Better for Prediction [10.098410272203301]
  This paper argues that generating output tokens is more effective than using pooled representations for prediction tasks because token-level generation retains more mutual information.<n>We introduce PredGen, an end to end framework that (i) uses scheduled sampling to reduce exposure bias, and (ii) introduces a task adapter to convert the generated tokens into structured outputs.<n>Our results show that PredGen consistently outperforms standard baselines, demonstrating its effectiveness in structured prediction tasks.
  arXiv  Detail & Related papers  (2025-02-25T03:48:19Z)
• Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Induction [2.3020018305241337]
  Our research promotes learning more compact, accurate, and consistent explicit grammars, facilitating better interpretability. In unsupervised parsing benchmark tests, our method significantly improves performance while reducing bias toward overly simplistic parses.
  arXiv  Detail & Related papers  (2024-07-23T04:57:03Z)
• TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation [65.65530016765615]
  We propose a hierarchical predictive coding framework that captures multi-scale dependencies through three complementary learning objectives.<n> TokenUnify integrates random token prediction, next-token prediction, and next-all token prediction to create a comprehensive representational space.<n>We also introduce a large-scale EM dataset with 1.2 billion annotated voxels, offering ideal long-sequence visual data with spatial continuity.
  arXiv  Detail & Related papers  (2024-05-27T05:45:51Z)
• Sentence Representation Learning with Generative Objective rather than Contrastive Objective [86.01683892956144]
  We propose a novel generative self-supervised learning objective based on phrase reconstruction. Our generative learning achieves powerful enough performance improvement and outperforms the current state-of-the-art contrastive methods.
  arXiv  Detail & Related papers  (2022-10-16T07:47:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.