TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints

• URL: http://arxiv.org/abs/2605.13076v1
• Date: Wed, 13 May 2026 06:49:08 GMT
• Title: TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints
• Abstract summary: TruncProof is a novel grammar-constrained generation method for machine-readable outputs.<n>We show that TruncProof can be effectively combined with advanced decoding strategies, resulting in grammatically valid and semantically accurate outputs.
• Score: 0.7734726150561086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The LLM-based generation of machine-readable outputs such as JSON has attracted significant attention for integration with external systems. However, existing approaches cannot strictly enforce the maximum number of tokens to be generated, leading to infinite generation or truncated outputs that cause a system malfunction. To address this limitation, we propose TruncProof, a novel grammar-constrained generation method that enables LLMs to produce grammatically valid JSONs while adhering to a predefined token limit. By leveraging the properties of LL(1) parsers, TruncProof efficiently approximates the minimum number of tokens required to complete a grammatically valid output at each decoding step. Experiments on the Text-to-JSON instruction tasks demonstrate that TruncProof successfully generates syntactically correct outputs even under strict token constraints. Furthermore, we show that TruncProof can be effectively combined with advanced decoding strategies, resulting in outputs that are not only grammatically valid but also semantically accurate.

Related papers

• Stay Within Your Bounds: Distance-Guided Decoding for Guaranteed Context-Free Grammar Compliance [18.38520405734125]
  We propose a lookahead-guided decoding framework for context-free grammars based on pushdown automata.<n>The resulting decoder is syntactically sound: every output is accepted by the target grammar.
  arXiv  Detail & Related papers  (2026-08-28T11:44:53Z)
• Efficient Grammar-Constrained Decoding via Parser Stack Classification [27.415578917916054]
  Grammar-constrained decoding (GCD) can guarantee the syntactic validity of the generated output, by masking out tokens that violate rules specified by a context-free grammar.<n>We propose PSC, a novel grammar-constrained decoding method.<n> PSC can compute the complete mask by checking the stack exactly once per decoding step, with time independent of the vocabulary size.
  arXiv  Detail & Related papers  (2026-08-04T03:26:54Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation [0.0]
  Token-Oriented Object Notation (TOON) aims to replace as a serialization format for passing structured data to LLMs with significantly reduced token usage.<n>To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation and comparing plain generation vs structured output.
  arXiv  Detail & Related papers  (2026-02-08T11:58:03Z)
• Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs [56.47577824219207]
  We present a prompt-based strategy that compels an off-the-shelf large language model to generate exactly a desired number of tokens.<n>The prompt appends countdown markers and explicit counting rules so that the model "writes while counting"<n>On MT-Bench-LI, strict length compliance with GPT-4.1 leaps from below 30% under naive prompts to above 95% with our countdown prompt.
  arXiv  Detail & Related papers  (2025-08-19T13:12:01Z)
• Earley-Driven Dynamic Pruning for Efficient Structured Decoding [6.905994258490735]
  Large Language Models (LLMs) have shown remarkable capabilities, yet ensuring their outputs conform to strict structural or grammatical constraints remains challenging.<n>We propose a novel strategy based on the Earley algorithm that identifies and eliminates invalid or redundant Earley states in real-time.<n>We implement a new constrained decoding engine called Formatron which also incorporates existing optimizations.
  arXiv  Detail & Related papers  (2025-06-01T20:05:30Z)
• Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors [77.82885394684202]
  We propose textbfContrastive textbfParaphrase textbfAttack (CoPA), a training-free method that effectively deceives text detectors.<n>CoPA constructs an auxiliary machine-like word distribution as a contrast to the human-like distribution generated by large language models.<n>Our theoretical analysis suggests the superiority of the proposed attack.
  arXiv  Detail & Related papers  (2025-05-21T10:08:39Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• $\texttt{SEM-CTRL}$: Semantically Controlled Decoding [53.86639808659575]
  $texttSEM-CTRL$ is a unified approach that enforces rich context-sensitive constraints and task- and instance-specific semantics directly on an LLM decoder.<n>texttSEM-CTRL$ allows small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models.
  arXiv  Detail & Related papers  (2025-03-03T18:33:46Z)
• Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection [49.15148871877941]
  Next-token distribution outputs offer a theoretically appealing approach for detection of large language models (LLMs)<n>We propose the Perplexity Attention Weighted Network (PAWN), which uses the last hidden states of the LLM and positions to weight the sum of a series of features based on metrics from the next-token distribution across the sequence length.<n>PAWN shows competitive and even better performance in-distribution than the strongest baselines with a fraction of their trainable parameters.
  arXiv  Detail & Related papers  (2025-01-07T17:00:49Z)
• Grammar-Aligned Decoding [30.972850034752884]
  Large Language Models (LLMs) struggle with reliably generating highly structured outputs, such as program code, mathematical formulas, or well-formed markup. Constrained decoding approaches mitigate this problem by greedily restricting what tokens an LLM can output at each step to guarantee that the output matches a given constraint. In this paper, we demonstrate that GCD techniques can distort the LLM's distribution, leading to outputs that are grammatical but appear with likelihoods that are not proportional to the ones given by the LLM.
  arXiv  Detail & Related papers  (2024-05-31T17:39:15Z)
• Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning [27.59524153097858]
  grammar-constrained decoding (GCD) can be used to control the generation of large language models (LMs) GCD can serve as a unified framework for structured NLP tasks in general. We show that grammar-constrained LMs substantially outperform unconstrained LMs or even beat task-specific finetuned models.
  arXiv  Detail & Related papers  (2023-05-23T11:54:37Z)
• Reducing Sequence Length by Predicting Edit Operations with Large Language Models [50.66922361766939]
  This paper proposes predicting edit spans for the source text for local sequence transduction tasks. We apply instruction tuning for Large Language Models on the supervision data of edit spans. Experiments show that the proposed method achieves comparable performance to the baseline in four tasks.
  arXiv  Detail & Related papers  (2023-05-19T17:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.