Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models

• URL: http://arxiv.org/abs/2608.14797v1
• Date: Fri, 14 Aug 2026 18:08:03 GMT
• Title: Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models
• Abstract summary: Large language models (LLMs) and large vision-language models (LVLMs) have demonstrated impressive generative capabilities.<n> Ensuring their outputs align with user intent is still challenging.<n>Inference-time approaches like decoding methods offer a more efficient and scalable solution.
• Score: 63.149913132071966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) and large vision-language models (LVLMs) have demonstrated impressive generative capabilities, yet ensuring their outputs align with user intent is still challenging. While most existing approaches address this issue at the training stage, inference-time approaches like decoding methods offer a more efficient and scalable solution. Decoding methods control model generation by guiding token-level selection, performing sequence-level generation, or generating tokens in parallel to accelerate the process. In this survey, we identify three emerging paradigms from recent works on decoding methods for LLMs and LVLMs, provide a systematic review of these methods, highlight ongoing challenges, and discuss potential future research directions. Our goal is to underscore the efficiency and effectiveness of decoding methods and offer a practical view of their applications. Paper lists and more resources on decoding methods for LLMs and LVLMs can be found at https://github.com/wang2226/Awesome-LLM-Decoding.

Related papers

• A Survey on Diffusion Language Models [30.00199970146068]
  Diffusion Language Models (DLMs) are an alternative to the dominant autoregressive (AR) paradigm.<n>DLMs possess inherent advantages in reducing inference latency and capturing bidirectional context.<n>Recent advancements have allowed DLMs to show performance comparable to their autoregressive counterparts.
  arXiv  Detail & Related papers  (2025-08-14T17:47:22Z)
• Prompt engineering and framework: implementation to increase code reliability based guideline for LLMs [0.0]
  We introduce a prompt template designed to improve the quality and correctness of generated code snippets.<n>We demonstrate that our approach outperforms widely studied zero-shot and Chain-of-Thought (CoT) methods in terms of the Pass@k metric.
  arXiv  Detail & Related papers  (2025-03-19T18:33:08Z)
• Ensemble Learning for Large Language Models in Text and Code Generation: A Survey [6.041894045506043]
  This article reviews emerging ensemble approaches to enhance understanding, encourage further research, and promote practical implementation in both text and code generation.<n>We categorize Large Language Models into seven main methods - weight merging, knowledge fusion, mixture-of-experts, output ensemble, routing, and cascading.<n>Our findings highlight key benefits such as improved diversity representation, enhanced output quality, and greater application flexibility.
  arXiv  Detail & Related papers  (2025-03-13T18:50:57Z)
• Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications [0.9105696129628794]
  Large Language Models (LLMs) have demonstrated their remarkable capabilities in numerous fields.<n>This survey focuses on how LLMs empower users, regardless of their technical background, to use human languages to automatically generate executable code.
  arXiv  Detail & Related papers  (2025-03-03T07:17:30Z)
• Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inference [82.557441484339]
  We propose a method to detect the knowledge boundary of Vision Large Language Models (VLLMs)<n> Experimental results on various types of Visual Question Answering datasets show that our method successfully depicts a VLLM's knowledge boundary.
  arXiv  Detail & Related papers  (2025-02-25T09:32:08Z)
• Reasoning Over the Glyphs: Evaluation of LLM's Decipherment of Rare Scripts [0.6144680854063939]
  We introduce a novel approach to construct a multimodal dataset of linguistic puzzles involving rare scripts.<n>We conduct experiments using prominent models, GPT-4o, Gemini, and Claude 3.5 Sonnet, on linguistic puzzles.<n>Our findings reveal the strengths and limitations of current AI methods in linguistic decipherment.
  arXiv  Detail & Related papers  (2025-01-29T17:24:19Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs)<n>We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation [35.10931307279044]
  This paper proposes Self-Evaluation Decoding, SED, a decoding method for enhancing model generation. It integrates speculation and evaluation steps into the decoding process, allowing LLMs to make more careful decisions.
  arXiv  Detail & Related papers  (2024-05-26T12:43:18Z)
• ST-LLM: Large Language Models Are Effective Temporal Learners [58.79456373423189]
  Large Language Models (LLMs) have showcased impressive capabilities in text comprehension and generation. How to effectively encode and understand videos in video-based dialogue systems remains to be solved. We propose ST-LLM, an effective video-LLM baseline with spatial-temporal sequence modeling inside LLM.
  arXiv  Detail & Related papers  (2024-03-30T10:11:26Z)
• A Thorough Examination of Decoding Methods in the Era of LLMs [72.65956436513241]
  Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. This paper provides a comprehensive and multifaceted analysis of various decoding methods within the context of large language models. Our findings reveal that decoding method performance is notably task-dependent and influenced by factors such as alignment, model size, and quantization.
  arXiv  Detail & Related papers  (2024-02-10T11:14:53Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.