RobustSpeechFlow: Learning Robust Text-to-Speech Trajectories via Augmentation-based Contrastive Flow Matching

• URL: http://arxiv.org/abs/2605.22083v1
• Date: Thu, 21 May 2026 07:22:28 GMT
• Title: RobustSpeechFlow: Learning Robust Text-to-Speech Trajectories via Augmentation-based Contrastive Flow Matching
• Abstract summary: RobustSpeechFlow is a training strategy that improves alignment by extending contrastive flow matching with length-preserving repeat and skip latent augmentations.<n>It reduces word error rate (WER) from 1.44 to 1.38 using only 0.06B parameters.<n>It delivers consistent intelligibility improvements across diverse speaker and prosody conditions.
• Score: 10.755921557009307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While flow-matching text-to-speech (TTS) achieves strong zero-shot speaker similarity and naturalness, it remains susceptible to content fidelity issues, particularly skip and repeat errors from imperfect alignment. We propose RobustSpeechFlow, a training strategy that improves alignment robustness by extending contrastive flow matching with length-preserving repeat and skip latent augmentations. Requiring no external aligners or preference data, our method directly penalizes realistic failure modes and readily integrates into existing pipelines. On Seed-TTS-eval, it reduces the word error rate (WER) from 1.44 to 1.38 using only 0.06B parameters. On our ZERO500 benchmark, it delivers consistent intelligibility improvements across diverse speaker and prosody conditions; at NFE=24, it reduces English character error rate (CER) from 0.48\% to 0.35\% and Korean CER from 0.81\% to 0.57\%. Audio samples: https://robustspeechflow.github.io/

Related papers

• LV-ROVER-MLT: Low-Resource Maltese OCR by Multi-Stream Voting [0.0]
  This paper reports results on a benchmark against a fine-tuned Tesseract baseline with a character error rate of 0.0234.<n> Ensemble recognition alone, scored under the same label convention as the baseline, improves character error rate by 44 percent to 0.01317.<n>A post-processing chain that aligns Tesseract's straight-quote and dash output to the benchmark's curly-quote convention, plus one stage that recovers misread diacritics, brings the full pipeline to a character error rate of 0.00700.
  arXiv  Detail & Related papers  (2026-06-30T22:58:41Z)
• Benchmarking Speech-to-Speech Translation Models [55.00303727199927]
  Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol.<n>We introduce a benchmarking framework integrating 46 metrics across eight dimensions.<n>We deploy it on 1,248 model-language configurations from FLEURS and CVSS.
  arXiv  Detail & Related papers  (2026-06-02T07:01:33Z)
• How Code Representation Shapes False-Positive Dynamics in Cross-Language LLM Vulnerability Detection [8.360057179027322]
  How code representation format shapes false positive behaviour in cross-language vulnerability detection remains poorly understood.<n>We systematically vary training intensity and code representation format, comparing raw source text with pruned Abstract Syntax Trees at both training time and inference time.<n>Cross-language FPR reflects the joint effect of training-time and inference-time representation, not either alone.
  arXiv  Detail & Related papers  (2026-04-30T11:01:03Z)
• Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR [9.626217175791572]
  We present a decoder-only Conformer for automatic speech recognition (ASR) that processes speech and text in a single stack without external speech encoders or pretrained large language models (LLM)<n>The model uses a modality-aware sparse mixture of experts (MoE): disjoint expert pools for speech and text with hard routing and top-1 selection, embedded in hybrid-causality Conformer blocks.<n>On Common Voice 16.1 with a single multilingual model across five languages, our approach reduces average WER from 12.2% to 10.6%.
  arXiv  Detail & Related papers  (2026-02-13T02:53:54Z)
• From Flows to Words: Can Zero-/Few-Shot LLMs Detect Network Intrusions? A Grammar-Constrained, Calibrated Evaluation on UNSW-NB15 [0.41998444721319217]
  Large Language Models (LLMs) can reason over natural-language inputs, but their role in intrusion detection without fine-tuning remains uncertain.<n>This study evaluates a promptonly approach by converting each network flow to a compact textual record and augmenting it with lightweight, domain-inspired flags.<n>We compare zero-shot, instruction-guided, and fewshot prompting to strong neural baselines under identical splits, reporting accuracy, precision, recall, F1, and macro scores.
  arXiv  Detail & Related papers  (2025-10-18T02:11:50Z)
• MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis [56.25862714128288]
  This paper introduces textitMegaTTS 3, a zero-shot text-to-speech (TTS) system featuring an innovative sparse alignment algorithm.<n>Specifically, we provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of alignment without limiting the search space.<n>Experiments demonstrate that MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible control over accent intensity.
  arXiv  Detail & Related papers  (2025-02-26T08:22:00Z)
• Phonology-Guided Speech-to-Speech Translation for African Languages [2.7624021966289605]
  We present a prosody-guided framework for speech-to-speech translation (S2ST)<n>We show that emphwithin-phylum language pairs exhibit 30--40% lower pause and over 3$times$ higher onset/offset correlation compared to cross-phylum pairs.<n>We also release a three-tier, transcript-free BLEU suite (M1--M3) that correlates strongly with human judgments.
  arXiv  Detail & Related papers  (2024-10-30T09:44:52Z)
• Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR [74.38242498079627]
  Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.<n>In this paper, SSL discrete speech features extracted from WavLM models are used as additional cross-utterance acoustic context features in Zipformer-Transducer ASR systems.
  arXiv  Detail & Related papers  (2024-09-13T13:01:09Z)
• Text Injection for Neural Contextual Biasing [57.589903308622745]
  This work proposes contextual text injection (CTI) to enhance contextual ASR. CTI with 100 billion text sentences can achieve up to 43.3% relative WER reduction from a strong neural biasing model.
  arXiv  Detail & Related papers  (2024-06-05T04:20:17Z)
• Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments [49.38965743465124]
  This paper introduces a streaming Transformer-Transducer that jointly generates automatic speech recognition (ASR) and speech translation (ST) outputs using a single decoder. Experiments in monolingual and multilingual settings demonstrate that our approach achieves the best quality-latency balance.
  arXiv  Detail & Related papers  (2023-07-07T02:26:18Z)
• CTC-based Non-autoregressive Speech Translation [51.37920141751813]
  We investigate the potential of connectionist temporal classification for non-autoregressive speech translation. We develop a model consisting of two encoders that are guided by CTC to predict the source and target texts. Experiments on the MuST-C benchmarks show that our NAST model achieves an average BLEU score of 29.5 with a speed-up of 5.67$times$.
  arXiv  Detail & Related papers  (2023-05-27T03:54:09Z)
• You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation [59.31769998728787]
  We build our TTS system on an ASR training database and then extend the data with synthesized speech to train a recognition model. Our system establishes a competitive result for end-to-end ASR trained on LibriSpeech train-clean-100 set with WER 4.3% for test-clean and 13.5% for test-other.
  arXiv  Detail & Related papers  (2020-05-14T17:24:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.