Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs

• URL: http://arxiv.org/abs/2605.12460v1
• Date: Tue, 12 May 2026 17:47:41 GMT
• Title: Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs
• Abstract summary: We show that models can be unblocked by switching from instruction-tuning for sequential message formats to instruction-tuning for multiple, parallel streams of computation.<n>Every forward pass of the language model then simultaneously reads from multiple input streams and generates tokens in multiple output streams, all of which causally depend on earlier timesteps.
• Score: 35.566195808249695
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The continued improvements in language model capability have unlocked their widespread use as drivers of autonomous agents, for example in coding or computer use applications. However, the core of these systems has not changed much since early instruction-tuned models like ChatGPT. Even advanced AI agents function on message exchange formats, successively exchanging messages with users, systems, with itself (i.e. chain-of-thought) and tools in a single stream of computation. This bottleneck to a single stream in chat models leads to a number of limitations: the agent cannot act (generate output) while reading, and in reverse, cannot react to new information while writing. Similarly, the agent cannot act while thinking and cannot think while reading or acting on information. In this work, we show that models can be unblocked by switching from instruction-tuning for sequential message formats to instruction-tuning for multiple, parallel streams of computation, splitting each role into a separate stream. Every forward pass of the language model then simultaneously reads from multiple input streams and generates tokens in multiple output streams, all of which causally depend on earlier timesteps. We argue that this data-driven change remedies a number of usability limitations as outlined above, improves model efficiency through parallelization, improves model security through better separation of concerns and can further improve model monitorability.

Related papers

• Continuous Autoregressive Language Models [56.49239051750678]
  We introduce Continuous Autoregressive Language Models (CALM)<n>CALM uses a high-fidelity autoencoder to compress a chunk of K tokens into a single continuous vector.<n>We develop a comprehensive likelihood-free framework that enables robust training, evaluation, and controllable sampling.
  arXiv  Detail & Related papers  (2025-10-31T17:58:11Z)
• Augmenting Multi-Agent Communication with State Delta Trajectory [44.68954502805962]
  We propose a new communication protocol that transfers both natural language tokens and token-wise state transition from one agent to another.<n>We find that the sequence of state changes in LLMs after generating each token can better reflect the information hidden behind the inference process.<n>The experimental results show that multi-agent systems with SDE achieve SOTA performance compared to other communication protocols.
  arXiv  Detail & Related papers  (2025-06-24T00:38:25Z)
• Sampling from Your Language Model One Byte at a Time [82.71473348639489]
  Tokenization can introduce distortion into the model's generations, known as the Prompt Boundary Problem (PBP)<n>We present an inference-time method to convert any autore LM with a BPE tokenizer into a character-level or byte-level LM.<n>Our method efficiently solves the PBP and is also able to unify the vocabularies of language models with different tokenizers.
  arXiv  Detail & Related papers  (2025-06-17T02:37:04Z)
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs [45.08958917457921]
  Large language models (LLMs) still struggle across tasks outside of high-resource languages.<n>In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce.
  arXiv  Detail & Related papers  (2025-05-23T20:28:31Z)
• Remote Timing Attacks on Efficient Language Model Inference [63.79839291641793]
  We show it is possible to exploit timing differences to mount a timing attack. We show how it is possible to learn the topic of a user's conversation with 90%+ precision. An adversary can leverage a boosting attack to recover PII placed in messages for open source systems.
  arXiv  Detail & Related papers  (2024-10-22T16:51:36Z)
• Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration [54.897493351694195]
  We propose a novel parallel decoding approach, namely textithidden transfer, which decodes multiple successive tokens simultaneously in a single forward pass. In terms of acceleration metrics, we outperform all the single-model acceleration techniques, including Medusa and Self-Speculative decoding.
  arXiv  Detail & Related papers  (2024-04-18T09:17:06Z)
• MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling [7.737755720567113]
  This paper proposes MuLTI, a highly accurate and efficient video-and-language understanding model. We design a Text-Guided MultiWay-Sampler based on adapt-pooling residual mapping and self-attention modules. We also propose a new pretraining task named Multiple Choice Modeling.
  arXiv  Detail & Related papers  (2023-03-10T05:22:39Z)
• Twist Decoding: Diverse Generators Guide Each Other [116.20780037268801]
  We introduce Twist decoding, a simple and general inference algorithm that generates text while benefiting from diverse models. Our method does not assume the vocabulary, tokenization or even generation order is shared.
  arXiv  Detail & Related papers  (2022-05-19T01:27:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.