Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective

• URL: http://arxiv.org/abs/2609.04489v1
• Date: Thu, 03 Sep 2026 21:23:04 GMT
• Title: Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective
• Abstract summary: We show how pause tokens reshape the training dynamics of fine-tuning.<n>Masked Boundary Pause consistently improves reasoning, achieving gains of up to 6 points on math and 2.5 points on code.<n>These results recast pause tokens as a training-dynamics intervention on the retention-adaptation trade-off.
• Score: 7.155337546555976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pause-token methods improve LLM reasoning by inserting special tokens into sequences. Prior work explains these gains through computational expressivity. However, there is relatively little investigation into the training dynamics of pause tokens. We explore how pause tokens reshape the training dynamics of fine-tuning. Two controlled pilots expose distinct asymmetries. On a synthetic continual-learning task, masked pauses overwrite a previously-learned distribution roughly 4x less at matched final adaptation (H1, mode retention); on a synthetic math-reasoning probe, the boundary-adjacent token comes to encode substantially more downstream-step information (H2, non-myopic compression). We formalize a training rule consistent with both - Masked Boundary Pause (MBP), pause tokens placed at reasoning-step boundaries with their loss masked. Across 1B-8B Qwen and Llama models, MBP consistently improves reasoning, achieving gains of up to 6 points on math and 2.5 points on code, while preserving general language understanding abilities. We further demonstrate that this mode-preserving strategy extend gains to GRPO. These results recast pause tokens as a training-dynamics intervention on the retention-adaptation trade-off, rather than merely an inference-time computation device.

Related papers

• SuperThoughts: Reasoning Tokens in Superposition [26.164480068360362]
  Long Chain-of-Thought (CoT) reasoning improves problem-solving but is computationally expensive due to sequential token generation.<n>We propose SuperThoughts, which compresses pairs of consecutive CoT tokens into single latent representations.<n>SuperThoughts achieves $sim$20--30% CoT length reduction while maintaining accuracy with minimal degradation.
  arXiv  Detail & Related papers  (2026-06-11T19:42:18Z)
• When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning [60.714416943748866]
  In single-stream autoregressive interfaces, the same tokens both update the model state and constitute an irreversible public commitment.<n>We introduce Side-by-Side (SxS) Interleaved Reasoning, which makes disclosure timing a controllable decision within standard autoregressive generation.
  arXiv  Detail & Related papers  (2026-05-05T02:59:58Z)
• Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization [74.91418266859297]
  Implicit process reward models (PRMs) provide fine-grained reward signals along the reasoning process.<n>PRMs suffer from a train-inference mismatch: training only constrains a sequence-level aggregate, whereas inference requires token-level scores to reflect local step quality.<n>We propose a novel Implicit Prefix-Value Reward Model (IPVRM), which directly learns a prefix-conditioned value function estimating the probability of eventual correctness.<n>We also propose Distribution-Level RL (DistRL), which computes TD advantages for both sampled tokens and high-probability candidate tokens.
  arXiv  Detail & Related papers  (2026-04-14T18:19:54Z)
• AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth [23.442686851761298]
  AdaPonderLM is a self-supervised recurrent language model that learns token-wise early exiting during pretraining.<n>AdaPonderLM reduces inference compute at about 10% while maintaining comparable language modeling perplexity and competitive downstream accuracy.
  arXiv  Detail & Related papers  (2026-03-02T14:28:16Z)
• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution [3.4666771782038652]
  Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency.<n>We introduce FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens.<n>We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning.
  arXiv  Detail & Related papers  (2025-10-18T10:22:13Z)
• Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space [38.50132130644233]
  Current approaches for scaling inference-time compute in computation transformers rely on training them to emit explicit chain-of-thought tokens before producing an answer.<n>Thoughtbubbles is a transformer variant that performs parallel adaptive computation in latent space by learning to fork or delete residual streams.<n>Thoughtbubbles outperforms both standard decoder LMs and non-adaptive parallel computation approaches on OpenWebText and peS2o perplexity and in zero-shot evaluations such as HellaSwag and LAMBADA.
  arXiv  Detail & Related papers  (2025-09-30T19:49:15Z)
• Soft Tokens, Hard Truths [17.640897774014707]
  This work introduces a scalable method to learn continuous CoTs via reinforcement learning (RL)<n>We use "soft" tokens: mixtures of tokens together with noise on the input embedding to provide RL exploration.<n>On math reasoning benchmarks with Llama and Qwen models up to 8B, training with continuous CoTs match discrete-token CoTs for pass@1 and surpass them for pass@32.
  arXiv  Detail & Related papers  (2025-09-23T15:43:47Z)
• Multipole Attention for Efficient Long Context Reasoning [64.94673641704289]
  Large Reasoning Models (LRMs) have shown promising accuracy improvements on complex problem-solving tasks.<n>LRMs need to generate long chain-of-thought reasoning in order to think before answering.<n>We introduce Multipole Attention, which accelerates autoregressive reasoning by only computing exact attention for the most important tokens.
  arXiv  Detail & Related papers  (2025-06-16T03:00:40Z)
• VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models [35.38573641029626]
  We introduce the novel task of Extreme Short Token Reduction, which aims to represent entire videos using a minimal set of discrete tokens.<n>On the Extreme Short Token Reduction task, our VQToken compresses sequences to just 0.07 percent of their original length while incurring only a 0.66 percent drop in accuracy on the NextQA-MC benchmark.
  arXiv  Detail & Related papers  (2025-03-21T09:46:31Z)
• Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions [14.85882273040068]
  Masked diffusion models (MDMs) have emerged as a promising alternative approach for generative modeling over discrete domains.<n>We show that adaptive inference can boost solving accuracy in pretrained MDMs from $7$% to $approx 90$%, even outperforming ARMs with $7times$ as many parameters.
  arXiv  Detail & Related papers  (2025-02-10T18:47:21Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)
• Toward Understanding BERT-Like Pre-Training for DNA Foundation Models [78.48760388079523]
  Existing pre-training methods for DNA sequences rely on direct adoptions of BERT pre-training from NLP. We introduce a novel approach called RandomMask, which gradually increases the task difficulty of BERT-like pre-training by continuously expanding its mask boundary. RandomMask achieves a staggering 68.16% in Matthew's correlation coefficient for Epigenetic Mark Prediction, a groundbreaking increase of 19.85% over the baseline.
  arXiv  Detail & Related papers  (2023-10-11T16:40:57Z)
• Revisiting Token Dropping Strategy in Efficient BERT Pretraining [102.24112230802011]
  Token dropping is a strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers. However, we empirically find that token dropping is prone to a semantic loss problem and falls short in handling semantic-intense tasks. Motivated by this, we propose a simple yet effective semantic-consistent learning method (ScTD) to improve the token dropping.
  arXiv  Detail & Related papers  (2023-05-24T15:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.