CoupleFace: Relation Matters for Face Recognition Distillation

• URL: http://arxiv.org/abs/2204.05502v1
• Date: Tue, 12 Apr 2022 03:25:42 GMT
• Title: CoupleFace: Relation Matters for Face Recognition Distillation
• Authors: Jiaheng Liu, Haoyu Qin, Yichao Wu, Jinyang Guo, Ding Liang, Ke Xu
• Abstract summary: We propose an effective face recognition distillation method called CoupleFace. We first propose to mine the informative mutual relations, and then introduce the Relation-Aware Distillation (RAD) loss to transfer the mutual relation knowledge of the teacher model to the student model. Based on our proposed CoupleFace, we have won the first place in the ICCV21 Masked Face Recognition Challenge (MS1M track)
• Score: 26.2626768462705
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Knowledge distillation is an effective method to improve the performance of a lightweight neural network (i.e., student model) by transferring the knowledge of a well-performed neural network (i.e., teacher model), which has been widely applied in many computer vision tasks, including face recognition. Nevertheless, the current face recognition distillation methods usually utilize the Feature Consistency Distillation (FCD) (e.g., L2 distance) on the learned embeddings extracted by the teacher and student models for each sample, which is not able to fully transfer the knowledge from the teacher to the student for face recognition. In this work, we observe that mutual relation knowledge between samples is also important to improve the discriminative ability of the learned representation of the student model, and propose an effective face recognition distillation method called CoupleFace by additionally introducing the Mutual Relation Distillation (MRD) into existing distillation framework. Specifically, in MRD, we first propose to mine the informative mutual relations, and then introduce the Relation-Aware Distillation (RAD) loss to transfer the mutual relation knowledge of the teacher model to the student model. Extensive experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed CoupleFace for face recognition. Moreover, based on our proposed CoupleFace, we have won the first place in the ICCV21 Masked Face Recognition Challenge (MS1M track).

Related papers

• Distilling Generative-Discriminative Representations for Very Low-Resolution Face Recognition [19.634712802639356]
  Very low-resolution face recognition is challenging due to loss of informative facial details in resolution degradation. We propose a generative-discriminative representation distillation approach that combines generative representation with cross-resolution aligned knowledge distillation. Our approach improves the recovery of the missing details in very low-resolution faces and achieves better knowledge transfer.
  arXiv  Detail & Related papers  (2024-09-10T09:53:06Z)
• Cross Architecture Distillation for Face Recognition [49.55061794917994]
  We develop an Adaptable Prompting Teacher network (APT) that integrates prompts into the teacher, enabling it to manage distillation-specific knowledge. Experiments on popular face benchmarks and two large-scale verification sets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2023-06-26T12:54:28Z)
• A Novel Self-Knowledge Distillation Approach with Siamese Representation Learning for Action Recognition [6.554259611868312]
  Self-knowledge distillation is an effective transfer of knowledge from a heavy network (teacher) to a small network (student) to boost students' performance. This paper introduces a novel Self-knowledge distillation approach via Siamese representation learning.
  arXiv  Detail & Related papers  (2022-09-03T01:56:58Z)
• On the benefits of knowledge distillation for adversarial robustness [53.41196727255314]
  We show that knowledge distillation can be used directly to boost the performance of state-of-the-art models in adversarial robustness. We present Adversarial Knowledge Distillation (AKD), a new framework to improve a model's robust performance.
  arXiv  Detail & Related papers  (2022-03-14T15:02:13Z)
• Teacher-Student Training and Triplet Loss to Reduce the Effect of Drastic Face Occlusion [15.44796695070395]
  We show that convolutional neural networks (CNNs) trained on fully-visible faces exhibit very low performance levels. While fine-tuning the deep learning models on occluded faces is extremely useful, we show that additional performance gains can be obtained by distilling knowledge from models trained on fully-visible faces. Our main contribution consists in a novel approach for knowledge distillation based on triplet loss, which generalizes across models and tasks.
  arXiv  Detail & Related papers  (2021-11-20T11:13:46Z)
• ProxylessKD: Direct Knowledge Distillation with Inherited Classifier for Face Recognition [84.49978494275382]
  Knowledge Distillation (KD) refers to transferring knowledge from a large model to a smaller one. In this work, we focus on its application in face recognition. We propose a novel method named ProxylessKD that directly optimize face recognition accuracy.
  arXiv  Detail & Related papers  (2020-10-31T13:14:34Z)
• Knowledge Distillation Meets Self-Supervision [109.6400639148393]
  Knowledge distillation involves extracting "dark knowledge" from a teacher network to guide the learning of a student network. We show that the seemingly different self-supervision task can serve as a simple yet powerful solution. By exploiting the similarity between those self-supervision signals as an auxiliary task, one can effectively transfer the hidden information from the teacher to the student.
  arXiv  Detail & Related papers  (2020-06-12T12:18:52Z)
• Residual Knowledge Distillation [96.18815134719975]
  This work proposes Residual Knowledge Distillation (RKD), which further distills the knowledge by introducing an assistant (A) In this way, S is trained to mimic the feature maps of T, and A aids this process by learning the residual error between them. Experiments show that our approach achieves appealing results on popular classification datasets, CIFAR-100 and ImageNet.
  arXiv  Detail & Related papers  (2020-02-21T07:49:26Z)
• Joint Deep Learning of Facial Expression Synthesis and Recognition [97.19528464266824]
  We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER. The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions. In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
  arXiv  Detail & Related papers  (2020-02-06T10:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.