Large Language Model Data Generation for Enhanced Intent Recognition in German Speech

• URL: http://arxiv.org/abs/2508.06277v1
• Date: Fri, 08 Aug 2025 12:54:09 GMT
• Title: Large Language Model Data Generation for Enhanced Intent Recognition in German Speech
• Authors: Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter,
• Abstract summary: Intent recognition (IR) for speech commands is essential for artificial intelligence (AI) assistant systems.<n>We propose a novel approach that combines an adapted Whisper ASR model, fine-tuned on elderly German speech.<n>We generate synthetic speech with a text-to-speech model and conduct extensive cross-dataset testing.
• Score: 14.788624194380825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intent recognition (IR) for speech commands is essential for artificial intelligence (AI) assistant systems; however, most existing approaches are limited to short commands and are predominantly developed for English. This paper addresses these limitations by focusing on IR from speech by elderly German speakers. We propose a novel approach that combines an adapted Whisper ASR model, fine-tuned on elderly German speech (SVC-de), with Transformer-based language models trained on synthetic text datasets generated by three well-known large language models (LLMs): LeoLM, Llama3, and ChatGPT. To evaluate the robustness of our approach, we generate synthetic speech with a text-to-speech model and conduct extensive cross-dataset testing. Our results show that synthetic LLM-generated data significantly boosts classification performance and robustness to different speaking styles and unseen vocabulary. Notably, we find that LeoLM, a smaller, domain-specific 13B LLM, surpasses the much larger ChatGPT (175B) in dataset quality for German intent recognition. Our approach demonstrates that generative AI can effectively bridge data gaps in low-resource domains. We provide detailed documentation of our data generation and training process to ensure transparency and reproducibility.

Related papers

• LESS: Large Language Model Enhanced Semi-Supervised Learning for Speech Foundational Models Using in-the-wild Data [5.021795689551854]
  LESS (Large Language Model Enhanced Semi-supervised Learning) is a versatile framework that uses Large Language Models (LLMs) to correct pseudo-labels generated on in-the-wild data.<n>Across Mandarin ASR and Spanish-to-English AST evaluations, LESS delivers consistent gains.<n>We have released the recipe as open source to facilitate further research in this area.
  arXiv  Detail & Related papers  (2025-06-05T03:00:04Z)
• Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages [0.43498389175652036]
  This study integrates traditional and novel language models with fine-tuned Whisper models to raise their performance in less commonly studied languages.<n>We demonstrate substantial improvements in word error rate, particularly in low-resource scenarios.<n>While the integration reliably benefits all model sizes, the extent of improvement varies, highlighting the importance of optimized language model parameters.
  arXiv  Detail & Related papers  (2025-03-30T18:03:52Z)
• DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data [84.01401439030265]
  Recent end-to-end speech language models (SLMs) have expanded upon the capabilities of large language models (LLMs)<n>We present a simple yet effective automatic process for creating speech-text pair data.<n>Our model demonstrates general capabilities for speech-related tasks without the need for speech instruction-tuning data.
  arXiv  Detail & Related papers  (2024-09-30T07:01:21Z)
• Speech Recognition Rescoring with Large Speech-Text Foundation Models [20.145389016219106]
  Large language models (LLM) have demonstrated the ability to understand human language by leveraging large amount of text data. Automatic speech recognition (ASR) systems are often limited by available transcribed speech data. Recent multi-modal large language models have demonstrated strong spoken language understanding.
  arXiv  Detail & Related papers  (2024-09-25T06:17:23Z)
• Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection [49.27067541740956]
  Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction.<n>Building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese.<n>We propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages.
  arXiv  Detail & Related papers  (2024-09-17T08:36:45Z)
• Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition [110.8431434620642]
  We introduce the generative speech transcription error correction (GenSEC) challenge. This challenge comprises three post-ASR language modeling tasks: (i) post-ASR transcription correction, (ii) speaker tagging, and (iii) emotion recognition. We discuss insights from baseline evaluations, as well as lessons learned for designing future evaluations.
  arXiv  Detail & Related papers  (2024-09-15T16:32:49Z)
• On decoder-only architecture for speech-to-text and large language model integration [59.49886892602309]
  Speech-LLaMA is a novel approach that effectively incorporates acoustic information into text-based large language models. We conduct experiments on multilingual speech-to-text translation tasks and demonstrate a significant improvement over strong baselines.
  arXiv  Detail & Related papers  (2023-07-08T06:47:58Z)
• Towards Language Modelling in the Speech Domain Using Sub-word Linguistic Units [56.52704348773307]
  We propose a novel LSTM-based generative speech LM based on linguistic units including syllables and phonemes. With a limited dataset, orders of magnitude smaller than that required by contemporary generative models, our model closely approximates babbling speech. We show the effect of training with auxiliary text LMs, multitask learning objectives, and auxiliary articulatory features.
  arXiv  Detail & Related papers  (2021-10-31T22:48:30Z)
• Meta-Transfer Learning for Code-Switched Speech Recognition [72.84247387728999]
  We propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting. Our model learns to recognize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code-switching data.
  arXiv  Detail & Related papers  (2020-04-29T14:27:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.