AFFAKT: A Hierarchical Optimal Transport based Method for Affective Facial Knowledge Transfer in Video Deception Detection

• URL: http://arxiv.org/abs/2412.08965v1
• Date: Thu, 12 Dec 2024 05:57:59 GMT
• Title: AFFAKT: A Hierarchical Optimal Transport based Method for Affective Facial Knowledge Transfer in Video Deception Detection
• Authors: Zihan Ji, Xuetao Tian, Ye Liu,
• Abstract summary: We propose a novel method called AFFAKT, which enhances the classification performance by transferring useful and correlated knowledge from a large facial expression dataset. Experimental results on two deception detection datasets demonstrate the superior performance of our proposed method.
• Score: 3.920204205770502
• License:
• Abstract: The scarcity of high-quality large-scale labeled datasets poses a huge challenge for employing deep learning models in video deception detection. To address this issue, inspired by the psychological theory on the relation between deception and expressions, we propose a novel method called AFFAKT in this paper, which enhances the classification performance by transferring useful and correlated knowledge from a large facial expression dataset. Two key challenges in knowledge transfer arise: 1) \textit{how much} knowledge of facial expression data should be transferred and 2) \textit{how to} effectively leverage transferred knowledge for the deception classification model during inference. Specifically, the optimal relation mapping between facial expression classes and deception samples is firstly quantified using proposed H-OTKT module and then transfers knowledge from the facial expression dataset to deception samples. Moreover, a correlation prototype within another proposed module SRKB is well designed to retain the invariant correlations between facial expression classes and deception classes through momentum updating. During inference, the transferred knowledge is fine-tuned with the correlation prototype using a sample-specific re-weighting strategy. Experimental results on two deception detection datasets demonstrate the superior performance of our proposed method. The interpretability study reveals high associations between deception and negative affections, which coincides with the theory in psychology.

Related papers

• T-KAER: Transparency-enhanced Knowledge-Augmented Entity Resolution Framework [2.8894038270224858]
  This paper introduces T-KAER, the Transparency-enhanced Knowledge-Augmented Entity Resolution framework. To enhance transparency, three Transparency-related Questions (T-Qs) have been proposed. To address the T-Qs, T-KAER is designed to improve transparency by documenting the entity resolution processes in log files.
  arXiv  Detail & Related papers  (2024-09-30T20:32:12Z)
• Knowledge-Aided Semantic Communication Leveraging Probabilistic Graphical Modeling [6.754511772924184]
  We propose a semantic communication approach based on probabilistic graphical model (PGM) We evaluate the importance of various semantic features and present a PGM-based compression algorithm designed to eliminate predictable portions of semantic information.
  arXiv  Detail & Related papers  (2024-08-08T14:50:48Z)
• Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning [19.962212551963383]
  Active Learning (AL) allows models to learn interactively from user feedback. This paper introduces a counterfactual data augmentation approach to AL.
  arXiv  Detail & Related papers  (2024-08-07T14:55:04Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Harnessing Diffusion Models for Visual Perception with Meta Prompts [68.78938846041767]
  We propose a simple yet effective scheme to harness a diffusion model for visual perception tasks. We introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception. Our approach achieves new performance records in depth estimation tasks on NYU depth V2 and KITTI, and in semantic segmentation task on CityScapes.
  arXiv  Detail & Related papers  (2023-12-22T14:40:55Z)
• CLIP-based Synergistic Knowledge Transfer for Text-based Person Retrieval [66.93563107820687]
  We introduce a CLIP-based Synergistic Knowledge Transfer (CSKT) approach for Person Retrieval (TPR) To explore the CLIP's knowledge on input side, we first propose a Bidirectional Prompts Transferring (BPT) module constructed by text-to-image and image-to-text bidirectional prompts and coupling projections. CSKT outperforms the state-of-the-art approaches across three benchmark datasets when the training parameters merely account for 7.4% of the entire model.
  arXiv  Detail & Related papers  (2023-09-18T05:38:49Z)
• Implicit Counterfactual Data Augmentation for Robust Learning [24.795542869249154]
  This study proposes an Implicit Counterfactual Data Augmentation method to remove spurious correlations and make stable predictions. Experiments have been conducted across various biased learning scenarios covering both image and text datasets.
  arXiv  Detail & Related papers  (2023-04-26T10:36:40Z)
• The Overlooked Classifier in Human-Object Interaction Recognition [82.20671129356037]
  We encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. We propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin.
  arXiv  Detail & Related papers  (2022-03-10T23:35:00Z)
• Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization [77.8735802150511]
  We propose a cross-sample adversarial debiasing (CSAD) method to remove the bias information misused by the target task. The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator. We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-08-11T21:17:02Z)
• Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation [53.49821324597837]
  Weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. We present a Context Decoupling Augmentation ( CDA) method to change the inherent context in which the objects appear. To validate the effectiveness of the proposed method, extensive experiments on PASCAL VOC 2012 dataset with several alternative network architectures demonstrate that CDA can boost various popular WSSS methods to the new state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-03-02T15:05:09Z)
• Transductive Zero-Shot Learning by Decoupled Feature Generation [30.664199050468472]
  We focus on the transductive setting, in which unlabelled visual data from unseen classes is available. We propose to decouple tasks of generating realistic visual features and translating semantic attributes into visual cues. We present a detailed ablation study to dissect the effect of our proposed decoupling approach, while demonstrating its superiority over the related state-of-the-art.
  arXiv  Detail & Related papers  (2021-02-05T16:17:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.