Covertly improving intelligibility with data-driven adaptations of speech timing

• URL: http://arxiv.org/abs/2603.30032v1
• Date: Tue, 31 Mar 2026 17:30:47 GMT
• Title: Covertly improving intelligibility with data-driven adaptations of speech timing
• Abstract summary: We take advantage of recent advancements in machine-generated speech allowing more precise control of speech rate.<n>We show that the temporal influence of speech rate prior to a target vowel contrast in fact manifests in a scissor-like pattern.<n>We build a data-driven text-to-speech algorithm that replicates this temporal structure on novel speech sequences.
• Score: 3.61940804174624
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human talkers often address listeners with language-comprehension challenges, such as hard-of-hearing or non-native adults, by globally slowing down their speech. However, it remains unclear whether this strategy actually makes speech more intelligible. Here, we take advantage of recent advancements in machine-generated speech allowing more precise control of speech rate in order to systematically examine how targeted speech-rate adjustments may improve comprehension. We first use reverse-correlation experiments to show that the temporal influence of speech rate prior to a target vowel contrast (ex. the tense-lax distinction) in fact manifests in a scissor-like pattern, with opposite effects in early versus late context windows; this pattern is remarkably stable both within individuals and across native L1-English listeners and L2-English listeners with French, Mandarin, and Japanese L1s. Second, we show that this speech rate structure not only facilitates L2 listeners' comprehension of the target vowel contrast, but that native listeners also rely on this pattern in challenging acoustic conditions. Finally, we build a data-driven text-to-speech algorithm that replicates this temporal structure on novel speech sequences. Across a variety of sentences and vowel contrasts, listeners remained unaware that such targeted slowing improved word comprehension. Strikingly, participants instead judged the common strategy of global slowing as clearer, even though it actually increased comprehension errors. Together, these results show that targeted adjustments to speech rate significantly aid intelligibility under challenging conditions, while often going unnoticed. More generally, this paper provides a data-driven methodology to improve the accessibility of machine-generated speech which can be extended to other aspects of speech comprehension and a wide variety of listeners and environments.

Related papers

• MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance [66.74042564585942]
  MOSS-Speech is a true speech-to-speech large language model that directly understands and generates speech without relying on text guidance.<n>Our work establishes a new paradigm for expressive and efficient end-to-end speech interaction.
  arXiv  Detail & Related papers  (2025-10-01T04:32:37Z)
• You Sound a Little Tense: L2 Tailored Clear TTS Using Durational Vowel Properties [3.61940804174624]
  We present the first text-to-speech (TTS) system tailored to second language (L2) speakers.<n>We use duration differences between American English tense (longer) and lax (shorter) vowels to create a "clarity mode" for Matcha-TTS.
  arXiv  Detail & Related papers  (2025-06-29T18:55:05Z)
• In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varieties [24.74769794165231]
  We introduce a scalable framework that allows for in-context learning (ICL) in Phi-4 Multimodal.<n>We find that as few as 12 example utterances at inference time reduce word error rates by a relative 19.7%.<n>Overall, we find that our novel ICL adaptation scheme reveals a similar performance profile to human listeners.
  arXiv  Detail & Related papers  (2025-05-20T20:20:37Z)
• Language-agnostic, automated assessment of listeners' speech recall using large language models [0.0]
  This research leverages modern large language models (LLMs) in native English speakers and native speakers of 10 other languages.<n>Participants listened to and freely recalled short stories (in quiet/clear and in babble noise) in their native language.<n>LLMs prompt engineering with semantic similarity analyses to score speech recall revealed sensitivity to known effects of temporal order, primacy/recency, and background noise.
  arXiv  Detail & Related papers  (2025-03-02T22:28:41Z)
• Continuous Speech Tokenizer in Text To Speech [27.057221389827735]
  We propose a continuous speech tokenizer named Cont-SPT, and a text-to-speech model based on continuous speech tokens.<n>Our results show that the speech language model based on the continuous speech tokenizer has better continuity and higher estimated Mean Opinion Scores (MoS)<n>This enhancement is attributed to better information preservation rate of the continuous speech tokenizer across both low and high frequencies in the frequency domain.
  arXiv  Detail & Related papers  (2024-10-22T15:02:37Z)
• Towards Unsupervised Speech Recognition Without Pronunciation Models [57.222729245842054]
  In this article, we tackle the challenge of developing ASR systems without paired speech and text corpora.<n>We experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling.<n>This innovative model surpasses the performance of previous unsupervised ASR models under the lexicon-free setting.
  arXiv  Detail & Related papers  (2024-06-12T16:30:58Z)
• ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading [65.88161811719353]
  This work develops a lightweight yet effective Text-to-Speech system, ContextSpeech. We first design a memory-cached recurrence mechanism to incorporate global text and speech context into sentence encoding. We construct hierarchically-structured textual semantics to broaden the scope for global context enhancement. Experiments show that ContextSpeech significantly improves the voice quality and prosody in paragraph reading with competitive model efficiency.
  arXiv  Detail & Related papers  (2023-07-03T06:55:03Z)
• MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition [51.412413996510814]
  We propose MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks. MixSpeech enhances speech translation in noisy environments, improving BLEU scores for four languages on AVMuST-TED by +1.4 to +4.2.
  arXiv  Detail & Related papers  (2023-03-09T14:58:29Z)
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data [54.733889961024445]
  We propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data. We evaluate the effectiveness of UniSpeech for cross-lingual representation learning on public CommonVoice corpus.
  arXiv  Detail & Related papers  (2021-01-19T12:53:43Z)
• Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion [60.808838088376675]
  We propose a VC system with explicit prosodic modelling and deep speaker embedding learning. A prosody corrector takes in phoneme embeddings to infer typical phoneme duration and pitch values. A conversion model takes phoneme embeddings and typical prosody features as inputs to generate the converted speech.
  arXiv  Detail & Related papers  (2020-11-03T13:08:53Z)
• Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training [40.71155396456831]
  Simultaneous speech-to-speech translation is widely useful but extremely challenging. It needs to generate target-language speech concurrently with the source-language speech, with only a few seconds delay. Current approaches accumulate latencies progressively when the speaker talks faster, and introduce unnatural pauses when the speaker talks slower. We propose Self-Adaptive Translation (SAT) which flexibly adjusts the length of translations to accommodate different source speech rates.
  arXiv  Detail & Related papers  (2020-10-20T06:02:15Z)
• On the Role of Style in Parsing Speech with Neural Models [25.442727974788255]
  We show that neural approaches facilitate using written text to improve parsing of spontaneous speech. We find an asymmetric degradation from read vs. spontaneous speech.
  arXiv  Detail & Related papers  (2020-10-08T22:44:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.