Large Audio Language Models for Spoofing-Aware Speaker Verification

• URL: http://arxiv.org/abs/2607.14753v1
• Date: Thu, 16 Jul 2026 09:38:45 GMT
• Title: Large Audio Language Models for Spoofing-Aware Speaker Verification
• Abstract summary: This work systematically evaluates large audio language models (LALMs) for spoofing-aware speaker verification (SASV)<n>Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not native to SASV.<n>These findings position LALMs as a promising and auditable foundation for unified SASV.
• Score: 2.5741280958081303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly address this threat through binary countermeasures (CMs) for deepfake detection or spoofing-aware speaker verification (SASV), where current systems are dominated by modular ASV-CM fusion and cascaded pipelines. Although large audio language models (LALMs) have shown promise on related audio tasks, including CM and ASV, their use for SASV remains unexplored, despite their capacity to produce natural-language rationales for auditing and robustness beyond discriminative predictions. This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization. Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not natively suited to SASV, but that task-specific adaptation closes this gap. We further find that competitive SASV performance can be achieved through several distinct routes. These findings position LALMs as a promising and auditable foundation for unified SASV, while clarifying where conventional cascade systems still lead.

Related papers

• DFKI-Speech System for WildSpoof Challenge: A robust framework for SASV In-the-Wild [6.7142637668069405]
  We propose a robust SASV framework in which a spoofing detector and a speaker verification network operate in tandem.<n>For speaker verification, we adapt a low-complexity convolutional neural network that fuses 2D and 1D features at multiple scales.<n> fixed cohort based AS Norm score normalization and model ensembling are used to further enhance the discriminative capability of the speaker verification system.
  arXiv  Detail & Related papers  (2026-02-02T16:27:04Z)
• Towards interfacing large language models with ASR systems using confidence measures and prompting [54.39667883394458]
  This work investigates post-hoc correction of ASR transcripts with large language models (LLMs) To avoid introducing errors into likely accurate transcripts, we propose a range of confidence-based filtering methods. Our results indicate that this can improve the performance of less competitive ASR systems.
  arXiv  Detail & Related papers  (2024-07-31T08:00:41Z)
• It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition [70.77292069313154]
  Large language models (LLMs) can be successfully used for generative error correction (GER) on top of the automatic speech recognition (ASR) output. In this work, we aim to overcome such a limitation by infusing acoustic information before generating the predicted transcription through a novel late fusion solution termed Uncertainty-Aware Dynamic Fusion (UADF)
  arXiv  Detail & Related papers  (2024-02-08T07:21:45Z)
• Generalizing Speaker Verification for Spoof Awareness in the Embedding Space [30.094557217931563]
  ASV systems can be spoofed using various types of adversaries. We propose a novel yet simple backend classifier based on deep neural networks. Experiments are conducted on the ASVspoof 2019 logical access dataset.
  arXiv  Detail & Related papers  (2024-01-20T07:30:22Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Leveraging Large Language Models for Exploiting ASR Uncertainty [16.740712975166407]
  Large language models must either rely on off-the-shelf automatic speech recognition systems for transcription, or be equipped with an in-built speech modality. We tackle speech-intent classification task, where a high word-error-rate can limit the LLM's ability to understand the spoken intent. We propose prompting the LLM with an n-best list of ASR hypotheses instead of only the error-prone 1-best hypothesis.
  arXiv  Detail & Related papers  (2023-09-09T17:02:33Z)
• Towards single integrated spoofing-aware speaker verification embeddings [63.42889348690095]
  This study aims to develop a single integrated spoofing-aware speaker verification embeddings. We analyze that the inferior performance of single SASV embeddings comes from insufficient amount of training data. Experiments show dramatic improvements, achieving a SASV-EER of 1.06% on the evaluation protocol of the SASV2022 challenge.
  arXiv  Detail & Related papers  (2023-05-30T14:15:39Z)
• Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion [88.34134732217416]
  This work focuses on fusion-based SASV solutions and proposes a multi-model fusion framework to leverage the power of multiple state-of-the-art ASV and CM models. The proposed framework vastly improves the SASV-EER from 8.75% to 1.17%, which is 86% relative improvement compared to the best baseline system in the SASV challenge.
  arXiv  Detail & Related papers  (2022-06-18T06:41:06Z)
• Spotting adversarial samples for speaker verification by neural vocoders [102.1486475058963]
  We adopt neural vocoders to spot adversarial samples for automatic speaker verification (ASV) We find that the difference between the ASV scores for the original and re-synthesize audio is a good indicator for discrimination between genuine and adversarial samples. Our codes will be made open-source for future works to do comparison.
  arXiv  Detail & Related papers  (2021-07-01T08:58:16Z)
• A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments [16.91453126121351]
  Speaker verification (SV) system should be robust to short speech segments, especially in noisy and reverberant environments. To meet these two requirements, we introduce feature pyramid module (FPM)-based multi-scale aggregation (MSA) and self-adaptive soft VAD (SAS-VAD) We combine SV, VAD, and SE models in a unified deep learning framework and jointly train the entire network in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-10-06T04:51:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.