Privacy-Enhancing Optical Embeddings for Lensless Classification

• URL: http://arxiv.org/abs/2211.12864v1
• Date: Wed, 23 Nov 2022 11:12:21 GMT
• Title: Privacy-Enhancing Optical Embeddings for Lensless Classification
• Abstract summary: We build a prototype from a low-cost LCD and Raspberry Pi components, for a total cost of around 100 USD. We show how the proposed system, through variable mask patterns, can thwart adversaries that attempt to invert the system.
• Score: 17.169529483306103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lensless imaging can provide visual privacy due to the highly multiplexed characteristic of its measurements. However, this alone is a weak form of security, as various adversarial attacks can be designed to invert the one-to-many scene mapping of such cameras. In this work, we enhance the privacy provided by lensless imaging by (1) downsampling at the sensor and (2) using a programmable mask with variable patterns as our optical encoder. We build a prototype from a low-cost LCD and Raspberry Pi components, for a total cost of around 100 USD. This very low price point allows our system to be deployed and leveraged in a broad range of applications. In our experiments, we first demonstrate the viability and reconfigurability of our system by applying it to various classification tasks: MNIST, CelebA (face attributes), and CIFAR10. By jointly optimizing the mask pattern and a digital classifier in an end-to-end fashion, low-dimensional, privacy-enhancing embeddings are learned directly at the sensor. Secondly, we show how the proposed system, through variable mask patterns, can thwart adversaries that attempt to invert the system (1) via plaintext attacks or (2) in the event of camera parameters leaks. We demonstrate the defense of our system to both risks, with 55% and 26% drops in image quality metrics for attacks based on model-based convex optimization and generative neural networks respectively. We open-source a wave propagation and camera simulator needed for end-to-end optimization, the training software, and a library for interfacing with the camera.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Lens Privacy Sealing: A New Benchmark and Method for Physical Privacy-Preserving Action Recognition [45.858644584022635]
  RGB camera-based surveillance systems enable human action recognition for public safety and healthcare.<n>Existing methods rely on post-capture algorithms, which fail to protect privacy during data acquisition.<n>We propose Lens Privacy Sealing (LPS), a simple hardware solution that physically obscures camera lenses with adjustable film.
  arXiv  Detail & Related papers  (2026-05-19T09:21:57Z)
• Sparse Transformer for Ultra-sparse Sampled Video Compressive Sensing [23.788589833081957]
  Digital cameras consume 0.1 microjoule per pixel to capture and encode video, resulting in a power usage of 20W for a 4K sensor operating at 30 fps.<n>We propose Ultra-Sparse Sampling (USS) regime, where at each spatial location, only one sub-frame is set to 1.<n>We then build a Digital Micro-mirror Device (DMD) encoding system to verify the effectiveness of our USS strategy.
  arXiv  Detail & Related papers  (2025-09-10T02:03:06Z)
• CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack [53.97547369215603]
  A projector-based adversarial attack aims to project light patterns onto scenes to deceive deep image classifiers.<n>It has potential applications in privacy protection and the development of more robust classifiers.<n>We introduce weighting-agnostic Projector-Based Adrial Attack (CAPAA) to address these issues.<n>CAPAA achieves both a higher attack success rate and greater stealthiness compared to existing baselines.
  arXiv  Detail & Related papers  (2025-06-01T12:17:49Z)
• Redundancy-Aware Camera Selection for Indoor Scene Neural Rendering [54.468355408388675]
  We build a similarity matrix that incorporates both the spatial diversity of the cameras and the semantic variation of the images. We apply a diversity-based sampling algorithm to optimize the camera selection. We also develop a new dataset, IndoorTraj, which includes long and complex camera movements captured by humans in virtual indoor environments.
  arXiv  Detail & Related papers  (2024-09-11T08:36:49Z)
• LenslessFace: An End-to-End Optimized Lensless System for Privacy-Preserving Face Verification [20.13507070548287]
  Lensless cameras encode light directly onto sensors, producing images that are not immediately recognizable. This paper presents an end-to-end optimization approach for privacy-preserving face verification directly on encoded lensless captures.
  arXiv  Detail & Related papers  (2024-06-06T14:50:15Z)
• MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems [22.494866649536018]
  Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios. We propose MC-NeRF, a method that enables joint optimization of both intrinsic and extrinsic parameters alongside NeRF.
  arXiv  Detail & Related papers  (2023-09-14T16:40:44Z)
• Human-imperceptible, Machine-recognizable Images [76.01951148048603]
  A major conflict is exposed relating to software engineers between better developing AI systems and distancing from the sensitive training data. This paper proposes an efficient privacy-preserving learning paradigm, where images are encrypted to become human-imperceptible, machine-recognizable'' We show that the proposed paradigm can ensure the encrypted images have become human-imperceptible while preserving machine-recognizable information.
  arXiv  Detail & Related papers  (2023-06-06T13:41:37Z)
• Learning rich optical embeddings for privacy-preserving lensless image classification [17.169529483306103]
  We exploit the unique multiplexing property of casting the optics as an encoder that produces learned embeddings directly at the camera sensor. We do so in the context of image classification, where we jointly optimize the encoder's parameters and those of an image classifier in an end-to-end fashion. Our experiments show that jointly learning the lensless optical encoder and the digital processing allows for lower resolution embeddings at the sensor, and hence better privacy as it is much harder to recover meaningful images from these measurements.
  arXiv  Detail & Related papers  (2022-06-03T07:38:09Z)
• Context-Aware Video Reconstruction for Rolling Shutter Cameras [52.28710992548282]
  In this paper, we propose a context-aware GS video reconstruction architecture. We first estimate the bilateral motion field so that the pixels of the two RS frames are warped to a common GS frame. Then, a refinement scheme is proposed to guide the GS frame synthesis along with bilateral occlusion masks to produce high-fidelity GS video frames.
  arXiv  Detail & Related papers  (2022-05-25T17:05:47Z)
• ConvMAE: Masked Convolution Meets Masked Autoencoders [65.15953258300958]
  Masked auto-encoding for feature pretraining and multi-scale hybrid convolution-transformer architectures can further unleash the potentials of ViT. Our ConvMAE framework demonstrates that multi-scale hybrid convolution-transformer can learn more discriminative representations via the mask auto-encoding scheme. Based on our pretrained ConvMAE models, ConvMAE-Base improves ImageNet-1K finetuning accuracy by 1.4% compared with MAE-Base.
  arXiv  Detail & Related papers  (2022-05-08T15:12:19Z)
• Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems [56.154190098338965]
  We show that our proposed time multiplexed coded aperture (TMCA) can be optimized end-to-end. TMCA induces better coded snapshots enabling superior reconstructions in two different applications: compressive light field imaging and hyperspectral imaging. This codification outperforms the state-of-the-art compressive imaging systems by more than 4dB in those applications.
  arXiv  Detail & Related papers  (2021-04-06T22:42:34Z)
• Adversarial Imaging Pipelines [28.178120782659878]
  We develop an attack that deceives a specific camera ISP while leaving others intact. We validate the proposed method using recent state-of-the-art automotive hardware ISPs.
  arXiv  Detail & Related papers  (2021-02-07T06:10:54Z)
• Infrastructure-based Multi-Camera Calibration using Radial Projections [117.22654577367246]
  Pattern-based calibration techniques can be used to calibrate the intrinsics of the cameras individually. Infrastucture-based calibration techniques are able to estimate the extrinsics using 3D maps pre-built via SLAM or Structure-from-Motion. We propose to fully calibrate a multi-camera system from scratch using an infrastructure-based approach.
  arXiv  Detail & Related papers  (2020-07-30T09:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.