On the Effect of Selfie Beautification Filters on Face Detection and Recognition

• URL: http://arxiv.org/abs/2110.08934v2
• Date: Wed, 20 Oct 2021 15:22:35 GMT
• Title: On the Effect of Selfie Beautification Filters on Face Detection and Recognition
• Authors: Pontus Hedman, Vasilios Skepetzis, Kevin Hernandez-Diaz, Josef Bigun, Fernando Alonso-Fernandez
• Abstract summary: Social media image filters modify the image contrast or illumination or occlude parts of the face with for example artificial glasses or animal noses. We develop a method to reconstruct the applied manipulation with a modified version of the U-NET segmentation network. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using a ResNet-34 network trained to recognize faces.
• Score: 53.561797148529664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Beautification and augmented reality filters are very popular in applications that use selfie images captured with smartphones or personal devices. However, they can distort or modify biometric features, severely affecting the capability of recognizing individuals' identity or even detecting the face. Accordingly, we address the effect of such filters on the accuracy of automated face detection and recognition. The social media image filters studied either modify the image contrast or illumination or occlude parts of the face with for example artificial glasses or animal noses. We observe that the effect of some of these filters is harmful both to face detection and identity recognition, specially if they obfuscate the eye or (to a lesser extent) the nose. To counteract such effect, we develop a method to reconstruct the applied manipulation with a modified version of the U-NET segmentation network. This is observed to contribute to a better face detection and recognition accuracy. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using a ResNet-34 network trained to recognize faces. We also evaluate if incorporating filtered images to the training set of machine learning approaches are beneficial for identity recognition. Our results show good recognition when filters do not occlude important landmarks, specially the eyes (identification accuracy >99%, EER<2%). The combined effect of the proposed approaches also allow to mitigate the effect produced by filters that occlude parts of the face, achieving an identification accuracy of >92% with the majority of perturbations evaluated, and an EER <8%. Although there is room for improvement, when neither U-NET reconstruction nor training with filtered images is applied, the accuracy with filters that severely occlude the eye is <72% (identification) and >12% (EER)

Related papers

• FaceFilterSense: A Filter-Resistant Face Recognition and Facial Attribute Analysis Framework [1.673834743879962]
  Fun selfie filters have come into tremendous mainstream use affecting the functioning of facial biometric systems. Current AR-based filters and filters which distort facial key points are in vogue recently and make the faces highly unrecognizable even to the naked eye. To mitigate these limitations, we aim to perform a holistic impact analysis of the latest filters and propose an user recognition model with the filtered images.
  arXiv  Detail & Related papers  (2024-04-12T07:04:56Z)
• Exploring Decision-based Black-box Attacks on Face Forgery Detection [53.181920529225906]
  Face forgery generation technologies generate vivid faces, which have raised public concerns about security and privacy. Although face forgery detection has successfully distinguished fake faces, recent studies have demonstrated that face forgery detectors are very vulnerable to adversarial examples.
  arXiv  Detail & Related papers  (2023-10-18T14:49:54Z)
• Has the Virtualization of the Face Changed Facial Perception? A Study of the Impact of Photo Editing and Augmented Reality on Facial Perception [7.532782211020641]
  We present the results of six surveys on familiarity with different styles of facial filters and ability to discern whether images are filtered. Our results demonstrate that faces modified with more traditional face filters are perceived similarly to unmodified faces. We discuss possible explanations for these results, including a societal adjustment to traditional photo editing techniques or the inherent differences in the different types of filters.
  arXiv  Detail & Related papers  (2023-03-01T16:09:11Z)
• Identity masking effectiveness and gesture recognition: Effects of eye enhancement in seeing through the mask [0.0]
  Face identity masking algorithms are designed to interfere with identification, while preserving information about facial actions. We evaluated the effectiveness of identity-masking algorithms based on Canny filters, applied with and without eye enhancement. We conclude that relatively simple, filter-based masking algorithms can be used in privacy protection without compromising action perception.
  arXiv  Detail & Related papers  (2023-01-20T03:10:19Z)
• LFW-Beautified: A Dataset of Face Images with Beautification and Augmented Reality Filters [53.180678723280145]
  We contribute with a database of facial images that includes several manipulations. It includes image enhancement filters (which mostly modify contrast and lightning) and augmented reality filters that incorporate items like animal noses or glasses. Each dataset contains 4,324 images of size 64 x 64, with a total of 34,592 images.
  arXiv  Detail & Related papers  (2022-03-11T17:05:10Z)
• Fun Selfie Filters in Face Recognition: Impact Assessment and Removal [13.715060479044167]
  This work investigates the impact of fun selfie filters on face recognition systems. Ten relevant fun selfie filters are selected to create a database. To mitigate such unwanted effects, a GAN-based selfie filter removal algorithm is proposed.
  arXiv  Detail & Related papers  (2022-02-12T09:12:31Z)
• End2End Occluded Face Recognition by Masking Corrupted Features [82.27588990277192]
  State-of-the-art general face recognition models do not generalize well to occluded face images. This paper presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks.
  arXiv  Detail & Related papers  (2021-08-21T09:08:41Z)
• Towards Face Encryption by Generating Adversarial Identity Masks [53.82211571716117]
  We propose a targeted identity-protection iterative method (TIP-IM) to generate adversarial identity masks. TIP-IM provides 95%+ protection success rate against various state-of-the-art face recognition models.
  arXiv  Detail & Related papers  (2020-03-15T12:45:10Z)
• Investigating the Impact of Inclusion in Face Recognition Training Data on Individual Face Identification [93.5538147928669]
  We audit ArcFace, a state-of-the-art, open source face recognition system, in a large-scale face identification experiment with more than one million distractor images. We find a Rank-1 face identification accuracy of 79.71% for individuals present in the model's training data and an accuracy of 75.73% for those not present.
  arXiv  Detail & Related papers  (2020-01-09T15:50:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.