Synthetic Audio Generation Framework for Air Traffic Control Speech Recognition

• URL: http://arxiv.org/abs/2606.21340v1
• Date: Fri, 19 Jun 2026 11:37:06 GMT
• Title: Synthetic Audio Generation Framework for Air Traffic Control Speech Recognition
• Abstract summary: We show that fine-tuning with either synthetic data alone, or a mix of real and synthetic data, significantly improves the word error rate over out-of-the-box and real data only baselines respectively.<n>Our experiments with the Whisper model on the ATCO2 corpus demonstrate that fine-tuning with either synthetic data alone, or a mix of real and synthetic data, significantly improves the word error rate over out-of-the-box and real data only baselines respectively.
• Score: 37.456341310281225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic Speech Recognition (ASR) systems, despite achieving remarkable accuracy in general-purpose domains with native speech (L1), struggle in domains like Air Traffic Control (ATC) due to strong channel noise, a presence of non-native (L2) English accents, and data scarcity. We propose a synthetic data generation pipeline with acoustical properties simulations specifically designed to address this lack of real data to improve recognition accuracy in the ATC domain. Our approach leverages a combination of neural generation techniques, including Text-to-Speech, Voice Conversion, L2-to-L1 accent conversion, and a novel controllable L1-to-L2 accent conversion framework built to simulate accented speech. Our experiments with the Whisper model on the ATCO2 corpus demonstrate that fine-tuning with either synthetic data alone, or a mix of real and synthetic data, significantly improves the word error rate over out-of-the-box and real data only baselines respectively.

Related papers

• How to Leverage Synthetic Speech for LLM-Based ASR Systems? [15.386863365540407]
  In regulated domains such as banking and healthcare, real speech is costly to collect and retain.<n>Yet a persistent distributional gap between synthetic and real data limits how far it can replace genuine recordings.<n>We show that convolving synthetic audio with room impulse responses (RIRs) narrows the gap by reproducing the acoustic irregularities of real recordings.
  arXiv  Detail & Related papers  (2026-06-27T17:57:27Z)
• ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis [3.1848820580333737]
  We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.<n>We propose ZeSTA, a simple domain-conditioned training framework that distinguishes real and synthetic speech.
  arXiv  Detail & Related papers  (2026-03-04T16:04:02Z)
• Stuttering-Aware Automatic Speech Recognition for Indonesian Language [0.04666493857924358]
  We propose a data augmentation framework that generates synthetic stuttered audio by injecting repetitions and prolongations into fluent text.<n>We apply this synthetic data to fine-tune a pre-trained Indonesian Whisper model using transfer learning.<n>Our experiments demonstrate that this targeted synthetic exposure consistently reduces recognition errors on stuttered speech while maintaining performance on fluent segments.
  arXiv  Detail & Related papers  (2026-01-07T09:21:12Z)
• Where are we in audio deepfake detection? A systematic analysis over generative and detection models [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark.<n>It provides a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content.<n>It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition [23.9811164130045]
  We propose a channel-aware data simulation method for robust automatic speech recognition training.<n>Our method harnesses the synergistic power of channel-extractive techniques and generative adversarial networks (GANs)<n>We evaluate our method on the challenging Hakka Across Taiwan (HAT) and Taiwanese Across Taiwan (TAT) corpora, achieving relative character error rate (CER) reductions of 20.02% and 9.64%, respectively.
  arXiv  Detail & Related papers  (2024-09-19T01:02:31Z)
• Speech collage: code-switched audio generation by collaging monolingual corpora [50.356820349870986]
  Speech Collage is a method that synthesizes CS data from monolingual corpora by splicing audio segments. We investigate the impact of generated data on speech recognition in two scenarios.
  arXiv  Detail & Related papers  (2023-09-27T14:17:53Z)
• Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation [76.13334392868208]
  Direct speech-to-speech translation (S2ST) models suffer from data scarcity issues. In this work, we explore self-supervised pre-training with unlabeled speech data and data augmentation to tackle this issue.
  arXiv  Detail & Related papers  (2022-04-06T17:59:22Z)
• Deliberation Model for On-Device Spoken Language Understanding [69.5587671262691]
  We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU) We show that our approach can significantly reduce the degradation when moving from natural speech to synthetic speech training.
  arXiv  Detail & Related papers  (2022-04-04T23:48:01Z)
• Speech recognition for air traffic control via feature learning and end-to-end training [8.755785876395363]
  We propose a new automatic speech recognition (ASR) system based on feature learning and an end-to-end training procedure for air traffic control (ATC) systems. The proposed model integrates the feature learning block, recurrent neural network (RNN), and connectionist temporal classification loss. Thanks to the ability to learn representations from raw waveforms, the proposed model can be optimized in a complete end-to-end manner.
  arXiv  Detail & Related papers  (2021-11-04T06:38:21Z)
• Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition [52.71604809100364]
  We propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech. Specifically, we feed original-noisy speech pairs simultaneously into the wav2vec 2.0 network. In addition to the existing contrastive learning task, we switch the quantized representations of the original and noisy speech as additional prediction targets.
  arXiv  Detail & Related papers  (2021-10-11T00:08:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.