Markovian Generation Chains in Large Language Models

• URL: http://arxiv.org/abs/2603.11228v1
• Date: Wed, 11 Mar 2026 18:54:17 GMT
• Title: Markovian Generation Chains in Large Language Models
• Abstract summary: We study how texts evolve when they are repeatedly processed by large language models (LLMs)<n>In this paper, we define this iterative inference process as Markovian generation chains.<n>We show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence.
• Score: 26.873145186799608
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior memory. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences over a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.

Related papers

• PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction [11.373080904038758]
  Large language models (LLMs) generate text by auto-regressively sampling the next token.<n>We present a functional approach to inverting a given LLM in a black-box setting, without auxiliary aids.<n>We train an explicit inverse language model entirely from scratch on data synthetically generated from the target LLM itself.
  arXiv  Detail & Related papers  (2026-07-31T13:01:43Z)
• Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing [28.646627695015646]
  Repetitive transformations can lead to stable configurations, known as attractors, including fixed points and limit cycles.<n>Applying this perspective to large language models (LLMs), which iteratively map input text to output text, provides a principled approach to characterizing long-term behaviors.<n>Successive paraphrasing serves as a compelling testbed for exploring such dynamics, as paraphrases re-express the same underlying meaning with linguistic variation.
  arXiv  Detail & Related papers  (2025-02-21T04:46:57Z)
• Idiosyncrasies in Large Language Models [54.26923012617675]
  We unveil and study idiosyncrasies in Large Language Models (LLMs)<n>We find that fine-tuning text embedding models on LLM-generated texts yields excellent classification accuracy.<n>We leverage LLM as judges to generate detailed, open-ended descriptions of each model's idiosyncrasies.
  arXiv  Detail & Related papers  (2025-02-17T18:59:02Z)
• Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation [60.493180081319785]
  We propose a systematic way to estimate the capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step.<n>Our work offers a comprehensive comparison of existing truncation sampling methods and serves as a practical user guideline for their parameter selection.
  arXiv  Detail & Related papers  (2024-08-24T14:14:32Z)
• Unraveling Text Generation in LLMs: A Stochastic Differential Equation Approach [3.4039202831583903]
  This paper explores the application of Differential Equations (SDE) to interpret the text generation process of Large Language Models (LLMs) such as GPT-4. We represent this generation process using SDE to capture both deterministic trends and perturbations. We fit these functions using neural networks and validate the model on real-world text corpora.
  arXiv  Detail & Related papers  (2024-08-17T15:30:27Z)
• Autoregressive Speech Synthesis without Vector Quantization [135.4776759536272]
  We present MELLE, a novel continuous-valued token based language modeling approach for text-to-speech synthesis (TTS)<n>MELLE autoregressively generates continuous mel-spectrogram frames directly from text condition.<n>MELLE mitigates robustness issues by avoiding the inherent flaws of sampling vector-quantized codes.
  arXiv  Detail & Related papers  (2024-07-11T14:36:53Z)
• Order-Independence Without Fine Tuning [18.020492646988746]
  We present Set-Based Prompting, a technique that guarantees the output of an LLM will not have order dependence on a specified set of sub-sequences.<n>Despite our inputs being out of distribution, the impact on expected accuracy is small, where the expectation is over the order of uniformly chosen shuffling of the candidate responses.
  arXiv  Detail & Related papers  (2024-06-04T16:09:13Z)
• Nearest Neighbor Speculative Decoding for LLM Generation and Attribution [87.3259169631789]
  Nearest Speculative Decoding (NEST) is capable of incorporating real-world text spans of arbitrary length into the LM generations and providing attribution to their sources.<n>NEST significantly enhances the generation quality and attribution rate of the base LM across a variety of knowledge-intensive tasks.<n>In addition, NEST substantially improves the generation speed, achieving a 1.8x speedup in inference time when applied to Llama-2-Chat 70B.
  arXiv  Detail & Related papers  (2024-05-29T17:55:03Z)
• Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
  Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus. Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
  arXiv  Detail & Related papers  (2023-10-17T03:21:43Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• A Simple Explanation for the Phase Transition in Large Language Models with List Decoding [3.898689841227059]
  We show that large language models (LLM) exhibit emergent abilities that are not present in small models. We use a list decoder that keeps a list of candidate sequences at each step and defers the generation of the output sequence at the end.
  arXiv  Detail & Related papers  (2023-03-23T09:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.