LenslessFace: An End-to-End Optimized Lensless System for Privacy-Preserving Face Verification

• URL: http://arxiv.org/abs/2406.04129v1
• Date: Thu, 6 Jun 2024 14:50:15 GMT
• Title: LenslessFace: An End-to-End Optimized Lensless System for Privacy-Preserving Face Verification
• Abstract summary: Lensless cameras encode light directly onto sensors, producing images that are not immediately recognizable. This paper presents an end-to-end optimization approach for privacy-preserving face verification directly on encoded lensless captures.
• Score: 20.13507070548287
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Lensless cameras, innovatively replacing traditional lenses for ultra-thin, flat optics, encode light directly onto sensors, producing images that are not immediately recognizable. This compact, lightweight, and cost-effective imaging solution offers inherent privacy advantages, making it attractive for privacy-sensitive applications like face verification. Typical lensless face verification adopts a two-stage process of reconstruction followed by verification, incurring privacy risks from reconstructed faces and high computational costs. This paper presents an end-to-end optimization approach for privacy-preserving face verification directly on encoded lensless captures, ensuring that the entire software pipeline remains encoded with no visible faces as intermediate results. To achieve this, we propose several techniques to address unique challenges from the lensless setup which precludes traditional face detection and alignment. Specifically, we propose a face center alignment scheme, an augmentation curriculum to build robustness against variations, and a knowledge distillation method to smooth optimization and enhance performance. Evaluations under both simulation and real environment demonstrate our method outperforms two-stage lensless verification while enhancing privacy and efficiency. Project website: \url{lenslessface.github.io}.

Related papers

• A Lightweight Face Quality Assessment Framework to Improve Face Verification Performance in Real-Time Screening Applications [0.0]
  Face image quality plays a critical role in determining the accuracy and reliability of face verification systems.<n>Low-quality face images, often caused by factors such as motion blur, poor lighting conditions, and extreme pose variations, significantly degrade the performance of face recognition models.<n>We propose a framework for automatic face quality assessment, which aims to pre-filter low-quality face images before they are passed to the verification pipeline.
  arXiv  Detail & Related papers  (2025-07-21T18:04:14Z)
• Privacy-preserving Optics for Enhancing Protection in Face De-identification [60.110274007388135]
  We propose a hardware-level face de-identification method to solve this vulnerability. We also propose an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input.
  arXiv  Detail & Related papers  (2024-03-31T19:28:04Z)
• Exploring Decision-based Black-box Attacks on Face Forgery Detection [53.181920529225906]
  Face forgery generation technologies generate vivid faces, which have raised public concerns about security and privacy. Although face forgery detection has successfully distinguished fake faces, recent studies have demonstrated that face forgery detectors are very vulnerable to adversarial examples.
  arXiv  Detail & Related papers  (2023-10-18T14:49:54Z)
• Attribute-preserving Face Dataset Anonymization via Latent Code Optimization [64.4569739006591]
  We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN. We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
  arXiv  Detail & Related papers  (2023-03-20T17:34:05Z)
• DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs [24.483597004603812]
  Close-up facial images captured at short distances often suffer from perspective distortion. We propose a simple yet effective method for correcting perspective distortions in a single close-up face.
  arXiv  Detail & Related papers  (2023-02-23T18:59:56Z)
• Privacy-Enhancing Optical Embeddings for Lensless Classification [17.169529483306103]
  We build a prototype from a low-cost LCD and Raspberry Pi components, for a total cost of around 100 USD. We show how the proposed system, through variable mask patterns, can thwart adversaries that attempt to invert the system.
  arXiv  Detail & Related papers  (2022-11-23T11:12:21Z)
• Learning rich optical embeddings for privacy-preserving lensless image classification [17.169529483306103]
  We exploit the unique multiplexing property of casting the optics as an encoder that produces learned embeddings directly at the camera sensor. We do so in the context of image classification, where we jointly optimize the encoder's parameters and those of an image classifier in an end-to-end fashion. Our experiments show that jointly learning the lensless optical encoder and the digital processing allows for lower resolution embeddings at the sensor, and hence better privacy as it is much harder to recover meaningful images from these measurements.
  arXiv  Detail & Related papers  (2022-06-03T07:38:09Z)
• OPOM: Customized Invisible Cloak towards Face Privacy Protection [58.07786010689529]
  We investigate the face privacy protection from a technology standpoint based on a new type of customized cloak. We propose a new method, named one person one mask (OPOM), to generate person-specific (class-wise) universal masks. The effectiveness of the proposed method is evaluated on both common and celebrity datasets.
  arXiv  Detail & Related papers  (2022-05-24T11:29:37Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Intelligent Frame Selection as a Privacy-Friendlier Alternative to Face Recognition [5.313209107051178]
  This study proposes a privacy-friendly alternative to large scale facial recognition. Instead of running facial recognition software on all video data, we propose to automatically extract a high quality snapshot of each detected person. This snapshot is then encrypted and access is only granted after legal authorization.
  arXiv  Detail & Related papers  (2021-01-19T09:31:42Z)
• Do We Need Depth in State-Of-The-Art Face Authentication? [8.755493949976492]
  We introduce a novel method that learns to recognize faces from stereo camera systems without the need to explicitly compute the facial surface or depth map. The raw face stereo images along with the location in the image from which the face is extracted allow the proposed CNN to improve the recognition task. We demonstrate that the suggested method outperforms both single-image and explicit depth based methods on large-scale benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T14:51:25Z)
• Face Anti-Spoofing by Learning Polarization Cues in a Real-World Scenario [50.36920272392624]
  Face anti-spoofing is the key to preventing security breaches in biometric recognition applications. Deep learning method using RGB and infrared images demands a large amount of training data for new attacks. We present a face anti-spoofing method in a real-world scenario by automatic learning the physical characteristics in polarization images of a real face.
  arXiv  Detail & Related papers  (2020-03-18T03:04:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.