LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering

• URL: http://arxiv.org/abs/2607.02763v1
• Date: Thu, 02 Jul 2026 21:00:52 GMT
• Title: LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering
• Abstract summary: Spoken Question Answering (SQA) remains largely focused on high-resource languages and carefully recorded speech.<n>This paper investigates whether text-to-speech (TTS) can provide task-specific training data for Luxembourgish SQA without requiring a large human-recorded QA corpus.
• Score: 8.948779410725903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spoken Question Answering (SQA) remains largely focused on high-resource languages and carefully recorded speech, limiting the reach of speech-LLM methods in low-resource settings. This paper investigates whether text-to-speech (TTS) can provide task-specific training data for Luxembourgish SQA without requiring a large human-recorded QA corpus. Starting from existing text-based QA resources, we translate questions into Luxembourgish, synthesize spoken questions with multiple TTS systems, and pair them with textual answers. We train a parameter-efficient SLAM-style architecture that connects a frozen Whisper encoder to frozen multilingual LLM backends through a learned projector and LoRA adapters. We compare MMS-TTS, Qwen3-TTS, and OmniVoice variants, including single-source corpora of about 48k questions and a 4TTS multi-source mix of approximately 230k questions. Evaluation on LLAMA-LB-Test with two real Luxembourgish speaker conditions shows that multi-source and voice-design-based synthetic training configurations yield the strongest SQA performance. The results also show that no-reference TTS quality scores do not monotonically predict downstream QA performance, indicating that synthetic speech must be evaluated as task-specific training data rather than only as natural-sounding audio.

Related papers

• SEAR: Segment-Evidence-Aware Routing for Weak-to-Strong Multilingual Speech MCQ [0.0]
  This paper describes our system for Task2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge.<n>A language model converts timestamped ASR into coherent event spans, which are expanded by a boundary margin and cropped from the original recording.<n>A text-only probe partitions the data into weak, text-answerable items used for supervised fine-tuning and strong, audio-dependent items used for reinforcement learning.
  arXiv  Detail & Related papers  (2026-09-10T10:38:19Z)
• Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision [36.3996368188181]
  Speech Large Language Models (LLMs) that understand and follow instructions in many languages are useful for real-world interaction.<n>Recent distillation-based approaches train performant English-only Speech LLMs using only annotated ASR data by aligning text and speech using only a lightweight projector.<n>We address this by introducing language-aware distillation using a query bank and a gating network that selects or mixes query tokens using a Q-Former projector.
  arXiv  Detail & Related papers  (2026-03-07T04:09:47Z)
• Fun-Audio-Chat Technical Report [71.07966678560291]
  temporal resolution between speech tokens (25Hz) and text tokens (3Hz) mitigates semantic information mismatchs and incurs high computational costs.<n>We introduce Fun-Audio-Chat, a Large-stage Spoken Speech-to-scale tasks.<n>Fun-Audio-Chat 8B and MoE 30BA3B achieve competitive performance on SpeechText and Speech-to-scale tasks.
  arXiv  Detail & Related papers  (2025-12-23T08:35:27Z)
• CCFQA: A Benchmark for Cross-Lingual and Cross-Modal Speech and Text Factuality Evaluation [26.054199546779696]
  CCFQA benchmark contains parallel speech-text factual questions across 8 languages.<n>Our experimental results demonstrate that current MLLMs still face substantial challenges on the CCFQA benchmark.<n>We propose a few-shot transfer learning strategy that effectively transfers the Question Answering (QA) capabilities of LLMs in English to multilingual Spoken Question Answering (SQA) tasks.
  arXiv  Detail & Related papers  (2025-08-10T11:09:41Z)
• Spoken question answering for visual queries [14.834200714168546]
  This work aims to create a system that enables user interaction through both speech and images.<n>The resulting multi-modal model has textual, visual, and spoken inputs and can answer spoken questions on images.
  arXiv  Detail & Related papers  (2025-05-29T10:06:48Z)
• SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs [12.60449414234283]
  SpokenNativQA is the first multilingual and culturally aligned spoken question-answering dataset.<n>The dataset comprises approximately 33,000 naturally spoken questions and answers in multiple languages.
  arXiv  Detail & Related papers  (2025-05-25T14:22:18Z)
• Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision [83.0622534215881]
  This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities.<n>Our pipeline consists of three main components: First, a modular framework enabling flexible configuration of various encoder-LLM-decoder architectures.<n>Second, a lightweight training strategy that pre-trains audio-language alignment on the state-of-the-art vision-language model Qwen2.5-VL.<n>Third, an audio synthesis pipeline that generates high-quality audio-text data from diverse real-world scenarios.
  arXiv  Detail & Related papers  (2025-02-26T17:26:36Z)
• VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning [64.56272011710735]
  We propose a novel single-stage joint speech-text SFT approach on the low-rank adaptation (LoRA) of the large language models (LLMs) backbone.<n>Compared to previous SpeechLMs with 7B or 13B parameters, our 3B model demonstrates superior performance across various speech benchmarks.
  arXiv  Detail & Related papers  (2024-10-23T00:36:06Z)
• QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning [65.35080911787882]
  This paper proposes a novel semi-supervised TTS framework, QS-TTS, to improve TTS quality with lower supervised data requirements. Two VQ-S3R learners provide profitable speech representations and pre-trained models for TTS. The results powerfully demonstrate the superior performance of QS-TTS, winning the highest MOS over supervised or semi-supervised baseline TTS approaches.
  arXiv  Detail & Related papers  (2023-08-31T20:25:44Z)
• DUAL: Textless Spoken Question Answering with Speech Discrete Unit Adaptive Learning [66.71308154398176]
  Spoken Question Answering (SQA) has gained research attention and made remarkable progress in recent years. Existing SQA methods rely on Automatic Speech Recognition (ASR) transcripts, which are time and cost-prohibitive to collect. This work proposes an ASR transcript-free SQA framework named Discrete Unit Adaptive Learning (DUAL), which leverages unlabeled data for pre-training and is fine-tuned by the SQA downstream task.
  arXiv  Detail & Related papers  (2022-03-09T17:46:22Z)
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering [65.124088336738]
  We propose a new Spoken Conversational Question Answering task (SCQA) SCQA aims at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. Our main objective is to build a QA system to deal with conversational questions both in spoken and text forms.
  arXiv  Detail & Related papers  (2020-10-18T05:53:39Z)
• LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition [148.43282526983637]
  We develop LRSpeech, a TTS and ASR system for languages with low data cost. We conduct experiments on an experimental language (English) and a truly low-resource language (Lithuanian) to verify the effectiveness of LRSpeech. We are currently deploying LRSpeech into a commercialized cloud speech service to support TTS on more rare languages.
  arXiv  Detail & Related papers  (2020-08-09T08:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.