How Transformers Learn to Plan via Multi-Token Prediction

• URL: http://arxiv.org/abs/2604.11912v1
• Date: Mon, 13 Apr 2026 18:04:09 GMT
• Title: How Transformers Learn to Plan via Multi-Token Prediction
• Abstract summary: Multi-token prediction (MTP) has emerged as a promising alternative to next-token prediction (NTP)<n>We show that MTP consistently outperforms NTP on both synthetic graph path-finding tasks and more realistic reasoning benchmarks.
• Score: 33.974020982252206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks. Multi-token prediction (MTP) has recently emerged as a promising alternative, yet its underlying mechanisms remain poorly understood. In this paper, we study how MTP facilitates reasoning, with a focus on planning. Empirically, we show that MTP consistently outperforms NTP on both synthetic graph path-finding tasks and more realistic reasoning benchmarks, such as Countdown and boolean satisfiability problems. Theoretically, we analyze a simplified two-layer Transformer on a star graph task. We prove that MTP induces a two-stage reverse reasoning process: the model first attends to the end node and then reconstructs the path by tracing intermediate nodes backward. This behavior arises from a gradient decoupling property of MTP, which provides a cleaner training signal compared to NTP. Ultimately, our results highlight how multi-token objectives inherently bias optimization toward robust and interpretable reasoning circuits.

Related papers

• P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling [9.276742450240961]
  Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text.<n>While Multi-Token Prediction (MTP) has emerged as a promising approach for accelerating inference, its potential is constrained by optimization instability when scaling to deeper look-ahead depth.<n>We propose textbfP-MTP, a framework that leverages textbf Multi-Token Prediction with a lightweight MTP module to scale the look-ahead depth.
  arXiv  Detail & Related papers  (2026-06-23T11:34:28Z)
• Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling [87.16803442525755]
  Reinforcement learning (RL) has become a key component in modern large language models, yet the rollout stage remains the key bottleneck in RL training pipelines.<n>Although Multi-Token Prediction (MTP) offers a natural solution to accelerate rollouts through speculative decoding, many studies have observed that MTP acceptance rates degrade significantly during RL training.<n>We present Bebop, a systematic study of MTP in LLM post-training, and offer practical recipes to integrate MTP into large-scale RL pipelines.
  arXiv  Detail & Related papers  (2026-06-10T17:36:45Z)
• Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration [61.46060073417047]
  Multi-Token Prediction (MTP) has been a widely adopted module in pretraining.<n>We show that the per-step effect of MTP on the RL objective can be decomposed into two terms: a first-order correlation and a second-order penalty.<n>We propose an adaptive scheme that tracks the optimal coefficient online via a log-probability proxy at negligible cost.
  arXiv  Detail & Related papers  (2026-05-27T09:07:06Z)
• Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement [12.77109338375446]
  Multi-Token Prediction (MTP) has shown promise in learning more structured representations.<n>We propose a novel method LSE-MTP, which anchors predictions to ground-truth hidden state trajectories.<n> Experiments on synthetic graphs and real-world Manhattan Taxi Ride show that LSE-MTP effectively bridges the gap between discrete tokens and continuous state representations.
  arXiv  Detail & Related papers  (2026-04-07T17:54:22Z)
• MTP-S2UT: Enhancing Speech-to-Speech Translation Quality with Multi-token Prediction [49.92201266421949]
  We introduce multi-token prediction (MTP) loss into speech-to-unit translation (S2UT) models.<n>We show that all MTP loss variants consistently improve the quality of S2UT translation.
  arXiv  Detail & Related papers  (2025-10-11T04:06:20Z)
• FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction [11.691960175716163]
  This paper introduces FastMTP, a method that improves multi-step draft quality by aligning MTP training with its inference pattern.<n>Our approach fine-tunes a single MTP head with position-shared weights on self-distilled data, enabling it to capture dependencies among consecutive future tokens.<n> Experimental results across seven diverse benchmarks demonstrate that FastMTP achieves an average of 2.03x speedup compared to standard next token prediction.
  arXiv  Detail & Related papers  (2025-09-16T07:36:26Z)
• Pre-Training Curriculum for Multi-Token Prediction in Language Models [2.8071268036220003]
  Multi-token prediction (MTP) is a recently proposed pre-training objective for language models.<n>We propose a curriculum learning strategy for MTP training, exploring two variants: a forward curriculum and a reverse curriculum.
  arXiv  Detail & Related papers  (2025-05-28T18:19:18Z)
• L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models [95.53699156138435]
  We propose leap multi-token prediction(L-MTP), an innovative token prediction method.<n>Unlike conventional MTP, L-MTP strategically skips over intermediate tokens, predicting non-sequential ones in a single forward pass.<n>We theoretically demonstrate the benefit of L-MTP in improving inference efficiency.
  arXiv  Detail & Related papers  (2025-05-23T05:59:46Z)
• Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models [74.40196814292426]
  We propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. GoT captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. We evaluate GoT's performance on a text-only reasoning task and a multimodal reasoning task.
  arXiv  Detail & Related papers  (2023-05-26T02:15:09Z)
• Learning Reasoning Strategies in End-to-End Differentiable Proving [50.9791149533921]
  Conditional Theorem Provers learn optimal rule selection strategy via gradient-based optimisation. We show that Conditional Theorem Provers are scalable and yield state-of-the-art results on the CLUTRR dataset.
  arXiv  Detail & Related papers  (2020-07-13T16:22:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.