The Role of Disfluencies in Speech Translation

• URL: http://arxiv.org/abs/2608.02138v1
• Date: Mon, 03 Aug 2026 12:26:39 GMT
• Title: The Role of Disfluencies in Speech Translation
• Abstract summary: Current speech translation systems tend to strip disfluencies like filled pauses and false starts rather than translate them.<n>We show this comes at a cost: disfluencies carry meaning that gets lost when speech is cleaned up.<n>We show inference-time decoding can mitigate this without retraining, and release the benchmark and code.
• Score: 74.9653116322058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current speech translation systems, including SpeechLLMs, are trained on cleaned text and tend to strip disfluencies like filled pauses and false starts rather than translate them. We show this comes at a cost: disfluencies carry meaning that gets lost when speech is cleaned up. To study this systematically, we introduce Uh-Mazing, a benchmark of human-translated, disfluency-annotated Switchboard speech covering English into eight target languages. Across these languages and several architectures, we find that false starts and self-repairs, not filled pauses or discourse markers, drive most of the translation-quality loss, and that models which fail to preserve a disfluency tend to omit it rather than mistranslate it. We show inference-time decoding can mitigate this without retraining, and release the benchmark and code.

Related papers

• Automatic Labelling of Speech Translation Errors [11.234683445082188]
  Speech Translation Error Labelling (STEL) is a methodology for evaluating confidence and quality estimation of speech translations.<n>We create an annotation protocol, a small authentic end-to-end evaluation dataset, and we analyse how existing text-only and speech-processing systems perform the STEL task.
  arXiv  Detail & Related papers  (2026-06-04T11:42:37Z)
• Mind the Pause: Disfluency-Aware Objective Tuning for Multilingual Speech Correction with LLMs [32.94544776067079]
  ASR transcripts often contain disfluencies, such as fillers, repetitions, and false starts.<n>Most existing approaches rely on classical models that focus on identifying disfluent tokens for removal.<n>We propose a multilingual correction pipeline where a sequence tagger first marks disfluent tokens, and these signals guide instruction fine-tuning of an LLM to rewrite transcripts into fluent text.
  arXiv  Detail & Related papers  (2026-05-12T15:11:36Z)
• Simultaneous Speech-to-Speech Translation Without Aligned Data [52.467808474293605]
  Simultaneous speech translation requires translating source speech into a target language in real-time.<n>We propose Hibiki-Zero, which eliminates the need for word-level alignments entirely.<n>Hibiki-Zero achieves state-of-the-art performance in translation accuracy, latency, voice transfer, and naturalness across five X-to-English tasks.
  arXiv  Detail & Related papers  (2026-02-11T17:41:01Z)
• Lost in Literalism: How Supervised Training Shapes Translationese in LLMs [51.04435855143767]
  Large language models (LLMs) have achieved remarkable success in machine translation.<n>However, translationese, characterized by overly literal and unnatural translations, remains a persistent challenge.<n>We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances.
  arXiv  Detail & Related papers  (2025-03-06T12:14:45Z)
• Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting [66.02718577386426]
  We provide a simple characterization of idiomatic translation and related issues. We conduct a synthetic experiment revealing a tipping point at which transformer-based machine translation models correctly default to idiomatic translations. To improve translation of natural idioms, we introduce two straightforward yet effective techniques.
  arXiv  Detail & Related papers  (2023-10-10T23:47:25Z)
• DisfluencyFixer: A tool to enhance Language Learning through Speech To Speech Disfluency Correction [50.51901599433536]
  DisfluencyFixer is a tool that performs speech-to-speech disfluency correction in English and Hindi. Our proposed system removes disfluencies from input speech and returns fluent speech as output.
  arXiv  Detail & Related papers  (2023-05-26T14:13:38Z)
• Assessing Evaluation Metrics for Speech-to-Speech Translation [9.670709690031885]
  Speech-to-speech translation combines machine translation with speech synthesis. How to automatically evaluate speech-to-speech translation is an open question which has not previously been explored.
  arXiv  Detail & Related papers  (2021-10-26T17:35:20Z)
• Bridging the Modality Gap for Speech-to-Text Translation [57.47099674461832]
  End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way. Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously. We propose a Speech-to-Text Adaptation for Speech Translation model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text.
  arXiv  Detail & Related papers  (2020-10-28T12:33:04Z)
• Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training [40.71155396456831]
  Simultaneous speech-to-speech translation is widely useful but extremely challenging. It needs to generate target-language speech concurrently with the source-language speech, with only a few seconds delay. Current approaches accumulate latencies progressively when the speaker talks faster, and introduce unnatural pauses when the speaker talks slower. We propose Self-Adaptive Translation (SAT) which flexibly adjusts the length of translations to accommodate different source speech rates.
  arXiv  Detail & Related papers  (2020-10-20T06:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.