Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speech

• URL: http://arxiv.org/abs/2509.16028v1
• Date: Fri, 19 Sep 2025 14:34:22 GMT
• Title: Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speech
• Authors: Sang Hoon Woo, Sehun Lee, Kang-wook Kim, Gunhee Kim,
• Abstract summary: Think-Verbalize-Speak is a framework that decouples reasoning from spoken delivery.<n>We also introduce ReVerT, a latency-efficient verbalizer based on incremental and asynchronous summarization.<n> Experiments across multiple benchmarks show that our method enhances speech naturalness and conciseness with minimal impact on reasoning.
• Score: 41.625380059502675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spoken dialogue systems increasingly employ large language models (LLMs) to leverage their advanced reasoning capabilities. However, direct application of LLMs in spoken communication often yield suboptimal results due to mismatches between optimal textual and verbal delivery. While existing approaches adapt LLMs to produce speech-friendly outputs, their impact on reasoning performance remains underexplored. In this work, we propose Think-Verbalize-Speak, a framework that decouples reasoning from spoken delivery to preserve the full reasoning capacity of LLMs. Central to our method is verbalizing, an intermediate step that translates thoughts into natural, speech-ready text. We also introduce ReVerT, a latency-efficient verbalizer based on incremental and asynchronous summarization. Experiments across multiple benchmarks show that our method enhances speech naturalness and conciseness with minimal impact on reasoning. The project page with the dataset and the source code is available at https://yhytoto12.github.io/TVS-ReVerT

Related papers

• MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance [66.74042564585942]
  MOSS-Speech is a true speech-to-speech large language model that directly understands and generates speech without relying on text guidance.<n>Our work establishes a new paradigm for expressive and efficient end-to-end speech interaction.
  arXiv  Detail & Related papers  (2025-10-01T04:32:37Z)
• Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models [80.75260664100644]
  Mini-Omni-Reasoner is a framework that enables reasoning within speech via a novel "Thinking-in-Speaking" formulation.<n>It interleaves silent reasoning tokens with spoken response tokens at the token level.<n>It achieves a +19.1% gain in arithmetic reasoning and +6.4% in contextual understanding, with shorter outputs and zero decoding latency.
  arXiv  Detail & Related papers  (2025-08-18T15:14:04Z)
• ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Models [70.56468982313834]
  We propose ProsodyLM, which introduces a simple tokenization scheme amenable to learning prosody.<n>We find that ProsodyLM can learn surprisingly diverse emerging prosody processing capabilities through pre-training alone.
  arXiv  Detail & Related papers  (2025-07-27T00:59:01Z)
• Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model [76.06585781346601]
  Speech language models (Speech LMs) enable end-to-end speech-text modelling within a single model.<n>The choice of speech-text jointly decoding paradigm plays a critical role in performance, efficiency, and alignment quality.
  arXiv  Detail & Related papers  (2025-06-04T23:53:49Z)
• IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilities [55.11130688075417]
  We introduce IntrinsicVoic,e an LLM designed with intrinsic real-time voice interaction capabilities. Our novelty architecture, GroupFormer, can reduce speech sequences to lengths comparable to text sequences. We construct a multi-turn speech-to-speech dialogue dataset named method-500k which includes nearly 500k turns of speech-to-speech dialogues.
  arXiv  Detail & Related papers  (2024-10-09T05:04:31Z)
• Recent Advances in Speech Language Models: A Survey [45.968078636811356]
  Speech Language Models (SpeechLMs) are end-to-end models that generate speech without converting from text.<n>This survey paper provides the first comprehensive overview of recent methodologies for constructing SpeechLMs.
  arXiv  Detail & Related papers  (2024-10-01T21:48:12Z)
• Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM [3.6950912517562435]
  We propose a method that implicitly internalizes ASR chain of thought into a speech LLM, enhancing its native speech understanding capabilities. Our approach reduces latency and improves the model's native understanding of speech, paving the way for more efficient and natural real-time audio interactions.
  arXiv  Detail & Related papers  (2024-09-25T20:59:12Z)
• Speech to Text Adaptation: Towards an Efficient Cross-Modal Distillation [15.225080891662675]
  Speech comprehension can benefit from inference of massive pre-trained language models. We experimentally verify our hypothesis that the knowledge could be shared from the top layer of the LM to a fully speech-based module.
  arXiv  Detail & Related papers  (2020-05-17T10:50:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.