Audio Deepfake Attribution: An Initial Dataset and Investigation

• URL: http://arxiv.org/abs/2208.10489v4
• Date: Sun, 17 Nov 2024 09:12:11 GMT
• Title: Audio Deepfake Attribution: An Initial Dataset and Investigation
• Authors: Xinrui Yan, Jiangyan Yi, Jianhua Tao, Jie Chen,
• Abstract summary: We design the first deepfake audio dataset for the attribution of audio generation tools, called Audio Deepfake Attribution (ADA) We propose the Class- Multi-Center Learning ( CRML) method for open-set audio deepfake attribution (OSADA) Experimental results demonstrate that the CRML method effectively addresses open-set risks in real-world scenarios.
• Score: 41.62487394875349
• License:
• Abstract: The rapid progress of deep speech synthesis models has posed significant threats to society such as malicious manipulation of content. This has led to an increase in studies aimed at detecting so-called deepfake audio. However, existing works focus on the binary detection of real audio and fake audio. In real-world scenarios such as model copyright protection and digital evidence forensics, binary classification alone is insufficient. It is essential to identify the source of deepfake audio. Therefore, audio deepfake attribution has emerged as a new challenge. To this end, we designed the first deepfake audio dataset for the attribution of audio generation tools, called Audio Deepfake Attribution (ADA), and conducted a comprehensive investigation on system fingerprints. To address the challenges of attribution of continuously emerging unknown audio generation tools in the real world, we propose the Class-Representation Multi-Center Learning (CRML) method for open-set audio deepfake attribution (OSADA). CRML enhances the global directional variation of representations, ensuring the learning of discriminative representations with strong intra-class similarity and inter-class discrepancy among known classes. Finally, the strong class discrimination capability learned from known classes is extended to both known and unknown classes. Experimental results demonstrate that the CRML method effectively addresses open-set risks in real-world scenarios. The dataset is publicly available at: https://zenodo.org/records/13318702, and https://zenodo.org/records/13340666.

Related papers

• SafeEar: Content Privacy-Preserving Audio Deepfake Detection [17.859275594843965]
  We propose SafeEar, a novel framework that aims to detect deepfake audios without relying on accessing the speech content within. Our key idea is to devise a neural audio into a novel decoupling model that well separates the semantic and acoustic information from audio samples. In this way, no semantic content will be exposed to the detector.
  arXiv  Detail & Related papers  (2024-09-14T02:45:09Z)
• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detection [17.285669984798975]
  This paper addresses the challenge of developing a robust audio-visual deepfake detection model. New generation algorithms are continually emerging, and these algorithms are not encountered during the development of detection methods. We propose a multi-stream fusion approach with one-class learning as a representation-level regularization technique.
  arXiv  Detail & Related papers  (2024-06-20T10:33:15Z)
• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection [53.063161380423715]
  Existing detection models have shown remarkable success in discriminating known deepfake audio, but struggle when encountering new attack types. We propose a continual learning approach called Radian Weight Modification (RWM) for audio deepfake detection.
  arXiv  Detail & Related papers  (2023-12-15T09:52:17Z)
• Text-to-feature diffusion for audio-visual few-shot learning [59.45164042078649]
  Few-shot learning from video data is a challenging and underexplored, yet much cheaper, setup. We introduce a unified audio-visual few-shot video classification benchmark on three datasets. We show that AV-DIFF obtains state-of-the-art performance on our proposed benchmark for audio-visual few-shot learning.
  arXiv  Detail & Related papers  (2023-09-07T17:30:36Z)
• SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection [54.74467470358476]
  This paper proposes a dataset for scene fake audio detection named SceneFake. A manipulated audio is generated by only tampering with the acoustic scene of an original audio. Some scene fake audio detection benchmark results on the SceneFake dataset are reported in this paper.
  arXiv  Detail & Related papers  (2022-11-11T09:05:50Z)
• Faked Speech Detection with Zero Prior Knowledge [2.407976495888858]
  We introduce a neural network method to develop a classifier that will blindly classify an input audio as real or mimicked. We propose a deep neural network following a sequential model that comprises three hidden layers, with alternating dense and drop out layers. We were able to get at least 94% correct classification of the test cases, as against the 85% accuracy in the case of human observers.
  arXiv  Detail & Related papers  (2022-09-26T10:38:39Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.