Spoken Language Identification with Pre-trained Models and Margin Loss

• URL: http://arxiv.org/abs/2605.01905v1
• Date: Sun, 03 May 2026 14:37:52 GMT
• Title: Spoken Language Identification with Pre-trained Models and Margin Loss
• Abstract summary: For the speaker-controlled spoken language identification task proposed in the TidyLang Challenge 2026, this paper proposes a language identification method based on pre-trained models and margin-based losses.<n>The proposed method adopts a pre-trained ECAPA-TDNN as the feature encoder and incorporates margin-based losses to enhance the discriminative ability of language representations.
• Score: 9.800066159312749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For the speaker-controlled spoken language identification task proposed in the TidyLang Challenge 2026, this paper proposes a language identification method based on pre-trained models and margin-based losses. The proposed method adopts a pre-trained ECAPA-TDNN as the feature encoder and incorporates margin-based losses to enhance the discriminative ability of language representations, thereby improving inter-class separability and reducing the interference of non-linguistic factors such as speaker characteristics. Experimental results on the Tidy-X dataset show that the proposed method achieves 85.95% macro accuracy and 90.96% micro accuracy on the language identification task and 17.08% equal error rate (EER) on the verification task. Compared with the official baseline, the macro accuracy improves by 45.7%, the micro accuracy improves by 15.2%, and the EER is reduced by approximately 50.8%, demonstrating the effectiveness of the proposed method. The code will be released at https://github.com/PunkMale/TidyLang2026.

Related papers

• Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM [0.0]
  We submit MéTRON-FR, a 125M GPT-2 pretrained on 92.47M words of French, to the BabyLM 2026 Strict track.<n>It scores 85.97 +/- 0.17% on QFrBLiMP and 62.80% on the BabyLM-weighted leaderboard.
  arXiv  Detail & Related papers  (2026-09-15T16:43:32Z)
• Robust Assamese Speech Recognition through Controlled Fine-Tuning of Whisper Models [1.4095958360608893]
  This paper presents a controlled, fine-tuned Whisper-based Assamese ASR system trained on the Mozilla Common Voice 24.0-Assamese corpus.<n>A hardware-aware optimized training pipeline is implemented for resource-constrained environments.
  arXiv  Detail & Related papers  (2026-07-19T09:54:52Z)
• Beyond Correctness: Evaluating Subjective Writing Preferences Across Cultures [87.75098311090642]
  Current preference learning methods achieve high accuracy on standard benchmarks but exhibit significant performance degradation when objective quality signals are removed.<n>We introduce WritingPreferenceBench, a dataset of 1,800 human-annotated preference pairs (1,200 English, 600 Chinese) across 8 creative writing genres.
  arXiv  Detail & Related papers  (2025-10-16T12:23:13Z)
• Robust Persian Digit Recognition in Noisy Environments Using Hybrid CNN-BiGRU Model [1.5566524830295307]
  This study addresses isolated spoken Persian digit recognition (zero to nine) under noisy conditions.<n>A hybrid model combining residual convolutional neural networks and bidirectional gated units (BiGRU) is proposed.<n> Experimental results demonstrate the model's effectiveness, achieving 98.53%, 96.10%, and 95.92% accuracy on training, validation, and test sets.
  arXiv  Detail & Related papers  (2024-12-14T15:11:42Z)
• Grammatical Error Correction for Low-Resource Languages: The Case of Zarma [8.40484790921164]
  Grammatical error correction aims to improve quality and readability of texts.<n>We present a study on GEC for Zarma, spoken by over five million in West Africa.<n>We compare three approaches: rule-based methods, machine translation (MT) models, and large language models.
  arXiv  Detail & Related papers  (2024-10-20T23:51:36Z)
• Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking [68.77659513993507]
  We present a simple and effective N-best re-ranking approach to improve multilingual ASR accuracy. Our results show spoken language identification accuracy improvements of 8.7% and 6.1%, respectively, and word error rates which are 3.3% and 2.0% lower on these benchmarks.
  arXiv  Detail & Related papers  (2024-09-27T03:31:32Z)
• Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method [108.56493934296687]
  We introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection.<n>We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text.
  arXiv  Detail & Related papers  (2024-09-23T07:55:35Z)
• Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents [39.92509218078164]
  We show for the first time, that in-context learning is sufficient to tackle multilingual TOD. We test our approach on the multilingual TOD dataset X-RiSAWOZ, which has 12 domains in Chinese, English, French, Korean, Hindi, and code-mixed Hindi-English.
  arXiv  Detail & Related papers  (2024-05-28T05:33:13Z)
• Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs [56.526095828316386]
  We propose a novel framework for adaptation with self-evaluation to improve the selective prediction performance of large language models (LLMs) We evaluate our method on a variety of question-answering (QA) datasets and show that it outperforms state-of-the-art selective prediction methods.
  arXiv  Detail & Related papers  (2023-10-18T03:34:59Z)
• DPCSpell: A Transformer-based Detector-Purificator-Corrector Framework for Spelling Error Correction of Bangla and Resource Scarce Indic Languages [2.5874041837241304]
  Spelling error correction is the task of identifying and rectifying misspelled words in texts.<n>Earlier efforts on spelling error correction in Bangla and resource-scarce Indic languages focused on rule-based, statistical, and machine learning-based methods.<n>We propose a novel detector-purificator-corrector, DPC based on denoising transformers by addressing previous issues.
  arXiv  Detail & Related papers  (2022-11-07T17:59:05Z)
• Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection [70.86672569101536]
  Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression. This paper investigates the use of prompt-based fine-tuning of PLMs that consistently uses AD classification errors as the training objective function.
  arXiv  Detail & Related papers  (2022-10-29T09:18:41Z)
• Semi-Supervised Speech Recognition via Local Prior Matching [42.311823406287864]
  Local prior matching is a semi-supervised objective that distills knowledge from a strong prior. We demonstrate that LPM is theoretically well-native, simple to implement, and superior to existing knowledge distillation techniques.
  arXiv  Detail & Related papers  (2020-02-24T16:07:11Z)
• Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition [58.105818353866354]
  We propose an improved recurrent neural network transducer (RNN-T) model with language bias to alleviate the problem. We use the language identities to bias the model to predict the CS points. This promotes the model to learn the language identity information directly from transcription, and no additional LID model is needed.
  arXiv  Detail & Related papers  (2020-02-19T12:01:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.