AdapTrack: Constrained Decoding without Distorting LLM's Output Intent

• URL: http://arxiv.org/abs/2510.17376v1
• Date: Mon, 20 Oct 2025 10:11:34 GMT
• Title: AdapTrack: Constrained Decoding without Distorting LLM's Output Intent
• Authors: Yongmin Li, Jia Li, Ge Li, Zhi Jin,
• Abstract summary: Language model-based code generation and completion tools can sometimes produce code that does not meet necessary constraints.<n> Constrained decoding techniques are developed to help the model generate code adhering to the constraints.<n>AdapTrack avoids distorting the output intent of the model, thereby producing results that are not only constraint-compliant but also semantically aligned with model's output intent.
• Score: 53.75852524070165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language model-based code generation and completion tools have been widely adopted, but they may sometimes produce code that does not meet necessary constraints, such as syntactic correctness or API existence. Constrained decoding techniques are developed to help the model generate code adhering to the constraints by greedily eliminating generation options that violate constraints at each step of the generation process. However, there is a severe limitation of constrained decoding, that it distorts the model's output intent, forcing it to produce code that may satisfy the constraint but does not match the development intent and is therefore incorrect. In response to this challenge, we propose AdapTrack. By incorporating backtracking into the generation process, AdapTrack avoids distorting the output intent of the model, thereby producing results that are not only constraint-compliant but also more semantically aligned with model's output intent. On our synthetic API completion dataset, AdapTrack can achieve up to 360.87% improvement compared to constrained decoding; on the real-world API completion dataset we collect that exhibits similar issues, AdapTrack can achieve up to 38.93% improvement over constrained decoding; in general code genration benchmarks, compared to constrained decoding, AdapTrack can achieve up to 7.84% improvement on HumanEval, and up to 6.42% improvement on MBPP. This indicates that, simply by better adhering to the model's output intent, AdapTrack can achieve significant improvements. We provide a theoretical proof that the distribution produced by AdapTrack aligns with the model's distribution given the generated tokens, thereby ensuring that the model's output intent is not distorted. Experiments on DSL problems show that, compared to existing methods, our approach can provide generation results that are more consistent with the language model's distribution.

Related papers

• Continuous Diffusion Models Can Obey Formal Syntax [37.090659119764176]
  Diffusion language models offer a promising alternative to autoregressive models due to their global, non-causal process.<n>We introduce a training-free method for steering continuous language models to satisfy formal syntactic constraints expressed using regular expressions.<n>Our approach constructs an analytic score estimating the probability that a latent state decodes to a valid string accepted by a given regular expression.
  arXiv  Detail & Related papers  (2026-02-12T22:55:05Z)
• IntentCoding: Amplifying User Intent in Code Generation [66.86062682889101]
  Large Language Models (LLMs) have shown strong capabilities in code generation, but their adherence to fine-grained user intent with multiple constraints remains a significant challenge.<n>We propose Intent-Amplified Code Generation (IntentCoding), a novel decoding strategy that enhances an LLM's ability to follow user intent.<n>IntentCoding captures the influence of user intent by masking out the intent, and applies a multi-strength ensemble mechanism to amplify the effect of user intent during generation.
  arXiv  Detail & Related papers  (2026-01-20T13:34:16Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• Continuous Speculative Decoding for Autoregressive Image Generation [27.308442169466975]
  Continuous visual autoregressive (AR) models have demonstrated promising performance in image generation.<n> speculative decoding has effectively accelerated discrete autoregressive inference.<n>This work addresses challenges from low acceptance rate, inconsistent output distribution, and modified distribution without analytic expression.
  arXiv  Detail & Related papers  (2024-11-18T09:19:15Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability [10.381674636820025]
  Large Language Models (LLMs) often produce outputs that lack consistency and reliability. We propose a novel game-theoretic approach to enhance consistency and reliability during the decoding stage of LLM output generation.
  arXiv  Detail & Related papers  (2024-10-01T20:46:10Z)
• FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking [57.53742155914176]
  API call generation is the cornerstone of large language models' tool-using ability. Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user's request. We propose an output-side optimization approach called FANTASE to address these limitations.
  arXiv  Detail & Related papers  (2024-07-18T23:44:02Z)
• Online Speculative Decoding [34.987825705622555]
  We introduce online speculative decoding to accelerate the inference of large language models. The main idea is to continuously update the (multiple) draft model(s) on observed user query data. We develop a prototype of online speculative decoding based on knowledge distillation and evaluate it using both synthetic and real query data.
  arXiv  Detail & Related papers  (2023-10-11T04:03:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.