Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems

• URL: http://arxiv.org/abs/2605.29676v1
• Date: Thu, 28 May 2026 09:38:13 GMT
• Title: Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems
• Abstract summary: Large language models in Agentic AI systems consume tool schemas and execution results as structured data.<n>Recent work proposes token-optimized alternatives such as TOON (Token-Oriented Object Notation) and TRON (Token Reduced Object Notation) as more compact replacements.<n>We evaluate TOON and TRON on four agentic benchmarks (BFCL, MCPBenchPP, MCP-Universe, StableTool) and five open-weight LLMs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models in Agentic AI systems consume tool schemas and execution results and emit tool invocations as structured data. The default language for that exchange, JSON, was designed for application-to-application interchange rather than token efficiency, so its structural elements impose substantial token overhead. Recent work proposes token-optimized alternatives such as TOON (Token-Oriented Object Notation) and TRON (Token Reduced Object Notation) as more compact replacements, but these formats have been evaluated only on isolated comprehension or generation tasks. Whether their token reductions hold inside end-to-end agentic loops therefore remains an open question. We evaluate TOON and TRON on four agentic benchmarks (BFCL, MCPToolBenchPP, MCP-Universe, StableToolBench) and five open-weight LLMs, decoupling input compression from output compression to measure comprehension and generation independently. TRON reduces tokens by up to 27% with accuracy within 14pp of the JSON baseline. TOON achieves up to 18% reduction at a similar 9pp accuracy cost, but additionally cascades on multi-turn parsing failures and collapses parallel tool-call output for most models.

Related papers

• Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs [0.08537603362277842]
  We show that neuro-symbolic decomposition enables lower-cost, non-reasoning large language models to generate complex workflow DAGs.<n>We show that the method improves judge validity by 24 percentage points and achieves statistical equivalence to a reasoning model's out-of-the-box quality.
  arXiv  Detail & Related papers  (2026-08-31T05:01:29Z)
• TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deployments [0.0]
  Production agent frameworks transmit tool schemas as MCP, a format designed for machine parsing, not for interpretation by language models.<n>For small models (4B-14B), this protocol mismatch accounts for the majority of tool-use failure at production sizes.<n>We present TSCG, a deterministic tool-schema compiler that resolves this mismatch at the API boundary.
  arXiv  Detail & Related papers  (2026-05-04T15:35:45Z)
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models [102.20309135516186]
  Cross-entropy (CE) training provides dense and scalable supervision for language models.<n>We introduce a feature-matching objective for language-model fine-tuning.<n>To optimize this objective efficiently, we propose energy-based fine-tuning.
  arXiv  Detail & Related papers  (2026-03-12T17:57:50Z)
• Towards Improved Sentence Representations using Token Graphs [41.412173502714225]
  We introduce GLOT, a structure-aware pooling module that reframes pooling as relational learning followed by aggregation.<n>On a diagnostic stress test where 90% of tokens are random distractors, GLOT maintains over 97% accuracy while baseline methods collapse.<n>It is competitive with state-of-the-art techniques on benchmarks like GLUE and MTEB with 20x fewer trainable parameters and speeds up the training time by over 100x compared with parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2026-03-03T09:00:01Z)
• Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation [0.0]
  Token-Oriented Object Notation (TOON) aims to replace as a serialization format for passing structured data to LLMs with significantly reduced token usage.<n>To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation and comparing plain generation vs structured output.
  arXiv  Detail & Related papers  (2026-02-08T11:58:03Z)
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs [57.217593337454026]
  TokenSqueeze is a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data.<n>We show that TokenSqueeze reduces token usage while maintaining accuracy on the MATH500 benchmark.
  arXiv  Detail & Related papers  (2025-11-17T10:38:56Z)
• TensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end Devices [19.897367559948336]
  This paper proposes a training-free token embedding compression approach using Train Decomposition (TTD)<n>We evaluate the extracted low-rank structures across compression ratio, language task performance, latency, and energy consumption on a typical low-end device, i.e. Raspberry Pi.
  arXiv  Detail & Related papers  (2025-06-16T14:09:43Z)
• Efficient Token Compression for Vision Transformer with Spatial Information Preserved [59.79302182800274]
  Token compression is essential for reducing the computational and memory requirements of transformer models.<n>We propose an efficient and hardware-compatible token compression method called Prune and Merge.
  arXiv  Detail & Related papers  (2025-03-30T14:23:18Z)
• SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator [65.62084602011596]
  Large Language Models (LLMs) have exhibited exceptional performance across a spectrum of natural language processing tasks.<n>We have identified a key pattern: certain seemingly meaningless separator tokens (i.e., punctuations) contribute disproportionately to attention scores compared to semantically meaningful tokens.<n>We introduce SepLLM, a plug-and-play framework that accelerates inference by compressing these segments and eliminating redundant tokens.
  arXiv  Detail & Related papers  (2024-12-16T18:58:57Z)
• ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models [32.95155349925248]
  We propose a modular paradigm ReWOO that detaches the reasoning process from external observations, thus significantly reducing token consumption. We show that ReWOO achieves 5x token efficiency and 4% accuracy improvement on HotpotQA, a multi-step reasoning benchmark. Our illustrative work offloads reasoning ability from 175B GPT3.5 into 7B LLaMA, demonstrating the significant potential for truly efficient and scalable ALM systems.
  arXiv  Detail & Related papers  (2023-05-23T00:16:48Z)
• ClusTR: Exploring Efficient Self-attention via Clustering for Vision Transformers [70.76313507550684]
  We propose a content-based sparse attention method, as an alternative to dense self-attention. Specifically, we cluster and then aggregate key and value tokens, as a content-based method of reducing the total token count. The resulting clustered-token sequence retains the semantic diversity of the original signal, but can be processed at a lower computational cost.
  arXiv  Detail & Related papers  (2022-08-28T04:18:27Z)
• Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP [52.25478388220691]
  Vision multi-layer perceptrons (MLPs) have shown promising performance in computer vision tasks. They use token-mixing layers to capture cross-token interactions, as opposed to the multi-head self-attention mechanism used by Transformers. We propose a new positional spacial gating unit (PoSGU) to efficiently encode the cross-token relations for token mixing.
  arXiv  Detail & Related papers  (2022-07-15T04:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.