How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?

• URL: http://arxiv.org/abs/2406.02483v1
• Date: Tue, 4 Jun 2024 16:51:42 GMT
• Title: How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?
• Authors: Tianchi Liu, Lin Zhang, Rohan Kumar Das, Yi Ma, Ruijie Tao, Haizhou Li,
• Abstract summary: countermeasures (CMs) trained on partially spoofed audio can effectively detect such spoofing. We utilize Grad-CAM and introduce a quantitative analysis metric to interpret CMs' decisions. We find that CMs prioritize the artifacts of transition regions created when concatenating bona fide and spoofed audio.
• Score: 53.58852794805362
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Partially manipulating a sentence can greatly change its meaning. Recent work shows that countermeasures (CMs) trained on partially spoofed audio can effectively detect such spoofing. However, the current understanding of the decision-making process of CMs is limited. We utilize Grad-CAM and introduce a quantitative analysis metric to interpret CMs' decisions. We find that CMs prioritize the artifacts of transition regions created when concatenating bona fide and spoofed audio. This focus differs from that of CMs trained on fully spoofed audio, which concentrate on the pattern differences between bona fide and spoofed parts. Our further investigation explains the varying nature of CMs' focus while making correct or incorrect predictions. These insights provide a basis for the design of CM models and the creation of datasets. Moreover, this work lays a foundation of interpretability in the field of partial spoofed audio detection that has not been well explored previously.

Related papers

• Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localization [60.899082019130766]
  We introduce a frame-level detection network (FDN) and a proposal refinement network (PRN) for audio temporal forgery detection and localization. FDN aims to mine informative inconsistency cues between real and fake frames to obtain discriminative features that are beneficial for roughly indicating forgery regions. PRN is responsible for predicting confidence scores and regression offsets to refine the coarse-grained proposals derived from the FDN.
  arXiv  Detail & Related papers  (2024-07-23T15:07:52Z)
• Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio [35.485350559012645]
  This paper defines Spoof Diarization as a novel task in the Partial Spoof (PS) scenario. It aims to determine what spoofed when, which includes locating spoof regions and clustering them according to different spoofing methods.
  arXiv  Detail & Related papers  (2024-06-12T02:23:57Z)
• Sentiment analysis in non-fixed length audios using a Fully Convolutional Neural Network [0.3495246564946556]
  A sentiment analysis method that is capable of accepting audio of any length, without being fixed a priori, is proposed. Mel spectrogram and Mel Frequency Cepstral Coefficients are used as audio description methods. A Fully Convolutional Neural Network architecture is proposed as a classifier.
  arXiv  Detail & Related papers  (2024-02-03T15:26:28Z)
• HM-Conformer: A Conformer-based audio deepfake detection system with hierarchical pooling and multi-level classification token aggregation methods [34.83806360076228]
  HM-Conformer is designed for sequence-to-sequence tasks. It can efficiently detect spoofing evidence by processing various sequence lengths and aggregating them. In experimental results, HM-Conformer achieved a 15.71% EER, showing competitive performance compared to recent systems.
  arXiv  Detail & Related papers  (2023-09-15T07:18:30Z)
• An Efficient Temporary Deepfake Location Approach Based Embeddings for Partially Spoofed Audio Detection [4.055489363682199]
  We propose a fine-grained partially spoofed audio detection method, namely Temporal Deepfake Location (TDL) Our approach involves two novel parts: embedding similarity module and temporal convolution operation. Our method outperform baseline models in ASVspoof 2019 Partial Spoof dataset and demonstrate superior performance even in the crossdataset scenario.
  arXiv  Detail & Related papers  (2023-09-06T14:29:29Z)
• DiffSED: Sound Event Detection with Denoising Diffusion [70.18051526555512]
  We reformulate the SED problem by taking a generative learning perspective. Specifically, we aim to generate sound temporal boundaries from noisy proposals in a denoising diffusion process. During training, our model learns to reverse the noising process by converting noisy latent queries to the groundtruth versions.
  arXiv  Detail & Related papers  (2023-08-14T17:29:41Z)
• Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection [54.20974251478516]
  We propose a continual learning algorithm for fake audio detection to overcome catastrophic forgetting. When fine-tuning a detection network, our approach adaptively computes the direction of weight modification according to the ratio of genuine utterances and fake utterances. Our method can easily be generalized to related fields, like speech emotion recognition.
  arXiv  Detail & Related papers  (2023-08-07T05:05:49Z)
• SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection [54.74467470358476]
  This paper proposes a dataset for scene fake audio detection named SceneFake. A manipulated audio is generated by only tampering with the acoustic scene of an original audio. Some scene fake audio detection benchmark results on the SceneFake dataset are reported in this paper.
  arXiv  Detail & Related papers  (2022-11-11T09:05:50Z)
• Cross-domain Adaptation with Discrepancy Minimization for Text-independent Forensic Speaker Verification [61.54074498090374]
  This study introduces a CRSS-Forensics audio dataset collected in multiple acoustic environments. We pre-train a CNN-based network using the VoxCeleb data, followed by an approach which fine-tunes part of the high-level network layers with clean speech from CRSS-Forensics.
  arXiv  Detail & Related papers  (2020-09-05T02:54:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.