Towards Unconstrained Audio Splicing Detection and Localization with Neural Networks

• URL: http://arxiv.org/abs/2207.14682v4
• Date: Fri, 3 May 2024 14:52:21 GMT
• Title: Towards Unconstrained Audio Splicing Detection and Localization with Neural Networks
• Authors: Denise Moussa, Germans Hirsch, Christian Riess,
• Abstract summary: Convincing forgeries can be created by combining various speech samples from the same person. Most existing detection algorithms for audio splicing use handcrafted features and make specific assumptions. We propose a Transformer sequence-to-sequence (seq2seq) network for splicing detection and localization.
• Score: 6.570712059945705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Freely available and easy-to-use audio editing tools make it straightforward to perform audio splicing. Convincing forgeries can be created by combining various speech samples from the same person. Detection of such splices is important both in the public sector when considering misinformation, and in a legal context to verify the integrity of evidence. Unfortunately, most existing detection algorithms for audio splicing use handcrafted features and make specific assumptions. However, criminal investigators are often faced with audio samples from unconstrained sources with unknown characteristics, which raises the need for more generally applicable methods. With this work, we aim to take a first step towards unconstrained audio splicing detection to address this need. We simulate various attack scenarios in the form of post-processing operations that may disguise splicing. We propose a Transformer sequence-to-sequence (seq2seq) network for splicing detection and localization. Our extensive evaluation shows that the proposed method outperforms existing dedicated approaches for splicing detection [3, 10] as well as the general-purpose networks EfficientNet [28] and RegNet [25].

Related papers

• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection [54.20974251478516]
  We propose a continual learning algorithm for fake audio detection to overcome catastrophic forgetting. When fine-tuning a detection network, our approach adaptively computes the direction of weight modification according to the ratio of genuine utterances and fake utterances. Our method can easily be generalized to related fields, like speech emotion recognition.
  arXiv  Detail & Related papers  (2023-08-07T05:05:49Z)
• TranssionADD: A multi-frame reinforcement based sequence tagging model for audio deepfake detection [11.27584658526063]
  The second Audio Deepfake Detection Challenge (ADD 2023) aims to detect and analyze deepfake speech utterances. We propose our novel TranssionADD system as a solution to the challenging problem of model robustness and audio segment outliers. Our best submission achieved 2nd place in Track 2, demonstrating the effectiveness and robustness of our proposed system.
  arXiv  Detail & Related papers  (2023-06-27T05:18:25Z)
• Synthetic Voice Detection and Audio Splicing Detection using SE-Res2Net-Conformer Architecture [2.9805017559176883]
  This paper extends the existing Res2Net by involving the recent Conformer block to further exploit the local patterns on acoustic features. Experimental results on ASVspoof 2019 database show that the proposed SE-Res2Net-Conformer architecture is able to improve the spoofing countermeasures performance. This paper also proposes to re-formulate the existing audio splicing detection problem.
  arXiv  Detail & Related papers  (2022-10-07T14:30:13Z)
• Deepfake audio detection by speaker verification [79.99653758293277]
  We propose a new detection approach that leverages only the biometric characteristics of the speaker, with no reference to specific manipulations. The proposed approach can be implemented based on off-the-shelf speaker verification tools. We test several such solutions on three popular test sets, obtaining good performance, high generalization ability, and high robustness to audio impairment.
  arXiv  Detail & Related papers  (2022-09-28T13:46:29Z)
• SoundDet: Polyphonic Sound Event Detection and Localization from Raw Waveform [48.68714598985078]
  SoundDet is an end-to-end trainable and light-weight framework for polyphonic moving sound event detection and localization. SoundDet directly consumes the raw, multichannel waveform and treats the temporal sound event as a complete sound-object" to be detected. A dense sound proposal event map is then constructed to handle the challenges of predicting events with large varying temporal duration.
  arXiv  Detail & Related papers  (2021-06-13T11:43:41Z)
• Multimodal Attention Fusion for Target Speaker Extraction [108.73502348754842]
  We propose a novel attention mechanism for multi-modal fusion and its training methods. Our proposals improve signal to distortion ratio (SDR) by 1.0 dB over conventional fusion mechanisms on simulated data.
  arXiv  Detail & Related papers  (2021-02-02T05:59:35Z)
• Cross-domain Adaptation with Discrepancy Minimization for Text-independent Forensic Speaker Verification [61.54074498090374]
  This study introduces a CRSS-Forensics audio dataset collected in multiple acoustic environments. We pre-train a CNN-based network using the VoxCeleb data, followed by an approach which fine-tunes part of the high-level network layers with clean speech from CRSS-Forensics.
  arXiv  Detail & Related papers  (2020-09-05T02:54:33Z)
• Identifying Audio Adversarial Examples via Anomalous Pattern Detection [4.556497931273283]
  We show that 2 of the recent and current state-of-the-art adversarial attacks on audio processing systems lead to higher-than-expected activation at some subset of nodes. We can detect these attacks with up to an AUC of 0.98 with no degradation in performance on benign samples.
  arXiv  Detail & Related papers  (2020-02-13T12:08:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.