ExpSpeech-Net: Multimodal Fusion of Expression and Speech for Deepfake Detection

• URL: http://arxiv.org/abs/2606.05760v1
• Date: Thu, 04 Jun 2026 06:41:36 GMT
• Title: ExpSpeech-Net: Multimodal Fusion of Expression and Speech for Deepfake Detection
• Abstract summary: The study introduces the ExpSpeech-Net deepfake detection (SqN-R-DFD) model, which utilizes SqueezeNet and RNN as its backbone.<n>The framework achieves 94.5% accuracy, precision of 99.3%, and F-measure of 96.8%, outperforming conventional methods.
• Score: 0.7734726150561088
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deepfake videos are increasingly challenging the credibility of online content. Many existing detection methodology relies on complex, resource-intensive models, which limit their practical use. The study introduces the ExpSpeech-Net deepfake detection (SqN-R-DFD) model, which utilizes SqueezeNet and RNN (Recurrent Neural Network) as its backbone, providing a lightweight and efficient deepfake detection framework that simultaneously analyzes facial expressions and speech patterns. The approach incorporates advanced feature extraction, such as ISLBT-based features for image and MPNCC for signals, along with a smart feature-selection strategy using SASMA (Sandpiper-Assisted Slime Mould Algorithm), ensuring optimal and balanced input to the detection models. By combining SqueezeNet and an RNN, subtle inconsistencies in deepfake videos are captured effectively. The framework achieves 94.5% accuracy, precision of 99.3%, and F-measure of 96.8%, outperforming conventional methods. This demonstrates that integrating multiple modalities with intelligent preprocessing and feature selection enables practical, real-time deepfake detection suitable for everyday applications.

Related papers

• SAND: Spatially Adaptive Network Depth for Fast Sampling of Neural Implicit Surfaces [91.71563221166417]
  Implicit neural representations are powerful for geometric modeling, but their practical use is often limited by the high computational cost.<n>We propose an efficient neural implicit geometry representation framework with spatially adaptive network depth.
  arXiv  Detail & Related papers  (2026-04-15T14:14:17Z)
• Enhancing Deepfake Detection using SE Block Attention with CNN [5.7494612007431805]
  We propose a lightweight convolution neural network (CNN) with squeeze and excitation block attention (SE) for Deepfake detection.<n>The model achieved an overall classification accuracy of 94.14% and AUC-ROC score of 0.985 on the Style GAN dataset.<n>Our proposed approach presents a promising avenue for combating the Deepfake challenge with minimal computational resources.
  arXiv  Detail & Related papers  (2025-06-12T13:29:26Z)
• Deepfake Detection with Optimized Hybrid Model: EAR Biometric Descriptor via Improved RCNN [1.1356542363919058]
  We introduce robust detection of subtle ear movements and shape changes to generate ear descriptors.<n>We also propose a novel optimized hybrid deepfake detection model that considers the ear biometric descriptors via enhanced RCNN.<n>Our proposed method outperforms traditional models such as CNN (Convolution Neural Network), SqueezeNet, LeNet, LinkNet, LSTM (Long Short-Term Memory), DFP (Deepfake Predictor), and ResNext+CNN+LSTM.
  arXiv  Detail & Related papers  (2025-03-16T07:01:29Z)
• Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networks [0.0]
  Deepfake detection aims to contrast the spread of deep-generated media that undermines trust in online content. We introduce a novel deepfake detection approach on images using Binary Neural Networks (BNNs) for fast inference with minimal accuracy loss.
  arXiv  Detail & Related papers  (2024-06-07T13:37:36Z)
• Unmasking Deepfake Faces from Videos Using An Explainable Cost-Sensitive Deep Learning Approach [0.0]
  Deepfake technology is widely used, which has led to serious worries about the authenticity of digital media. This study employs a resource-effective and transparent cost-sensitive deep learning method to effectively detect deepfake faces in videos.
  arXiv  Detail & Related papers  (2023-12-17T14:57:10Z)
• Global Context Aggregation Network for Lightweight Saliency Detection of Surface Defects [70.48554424894728]
  We develop a Global Context Aggregation Network (GCANet) for lightweight saliency detection of surface defects on the encoder-decoder structure. First, we introduce a novel transformer encoder on the top layer of the lightweight backbone, which captures global context information through a novel Depth-wise Self-Attention (DSA) module. The experimental results on three public defect datasets demonstrate that the proposed network achieves a better trade-off between accuracy and running efficiency compared with other 17 state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-22T06:19:11Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features [13.033517345182728]
  Deepfakes is a branch of malicious techniques that transplant a target face to the original one in videos. Previous efforts for Deepfakes videos detection mainly focused on appearance features, which have a risk of being bypassed by sophisticated manipulation. We propose an efficient and robust framework named LRNet for detecting Deepfakes videos through temporal modeling on precise geometric features.
  arXiv  Detail & Related papers  (2021-04-09T16:57:55Z)
• Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues [75.1731999380562]
  We present a learning-based method for detecting real and fake deepfake multimedia content. We extract and analyze the similarity between the two audio and visual modalities from within the same video. We compare our approach with several SOTA deepfake detection methods and report per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets.
  arXiv  Detail & Related papers  (2020-03-14T22:07:26Z)
• Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread [136.2224792151324]
  We propose a new deep learning algorithm for fast salient object detection. The proposed algorithm achieves competitive accuracy and high inference efficiency simultaneously with a single CPU thread.
  arXiv  Detail & Related papers  (2020-01-22T15:23:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.