Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices

• URL: http://arxiv.org/abs/2408.08529v1
• Date: Fri, 16 Aug 2024 04:57:21 GMT
• Title: Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices
• Authors: Kouki Horio, Kiyoshi Nishikawa, Hitoshi Kiya,
• Abstract summary: We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption.
• Score: 5.311735227179715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption. In contrast, the proposed encryption method using restricted random permutation matrices can provide a higher performance than the conventional ones.

Related papers

• Enabling Practical and Privacy-Preserving Image Processing [5.526464269029825]
  Homomorphic Encryption (FHE) enables computations on encrypted data, preserving confidentiality without the need for decryption. Traditional FHE methods often encrypt images by monolithic data blocks, instead of pixels. We propose and implement a pixel-level homomorphic encryption approach, iCHEETAH, based on the CKKS scheme.
  arXiv  Detail & Related papers  (2024-09-05T14:22:02Z)
• An Effective Approach to Scramble Multiple Diagnostic Imageries Using Chaos-Based Cryptography [0.0]
  We provide a chaotic system-based medical picture encryption method. The permutation based on plain image and chaotic keys is offered to shuffle the plain picture's pixels to other rows and columns. We analyze the chaotic behavior of the proposed system using various techniques and tests such as bifurcation plots, Lyapunov exponents, MSE, PSNR tests, and histogram analysis.
  arXiv  Detail & Related papers  (2024-05-02T05:18:46Z)
• Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer [6.476298483207895]
  We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT) The method allows us not only to train models and test with visually protected images but to also avoid the performance degradation caused from the use of encrypted images. A domain adaptation method is used to efficiently fine-tune ViT with encrypted images.
  arXiv  Detail & Related papers  (2024-01-10T12:46:31Z)
• Perceptual Image Compression with Cooperative Cross-Modal Side Information [53.356714177243745]
  We propose a novel deep image compression method with text-guided side information to achieve a better rate-perception-distortion tradeoff. Specifically, we employ the CLIP text encoder and an effective Semantic-Spatial Aware block to fuse the text and image features.
  arXiv  Detail & Related papers  (2023-11-23T08:31:11Z)
• PRO-Face S: Privacy-preserving Reversible Obfuscation of Face Images via Secure Flow [69.78820726573935]
  We name it PRO-Face S, short for Privacy-preserving Reversible Obfuscation of Face images via Secure flow-based model. In the framework, an Invertible Neural Network (INN) is utilized to process the input image along with its pre-obfuscated form, and generate the privacy protected image that visually approximates to the pre-obfuscated one.
  arXiv  Detail & Related papers  (2023-07-18T10:55:54Z)
• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• Human-imperceptible, Machine-recognizable Images [76.01951148048603]
  A major conflict is exposed relating to software engineers between better developing AI systems and distancing from the sensitive training data. This paper proposes an efficient privacy-preserving learning paradigm, where images are encrypted to become human-imperceptible, machine-recognizable'' We show that the proposed paradigm can ensure the encrypted images have become human-imperceptible while preserving machine-recognizable information.
  arXiv  Detail & Related papers  (2023-06-06T13:41:37Z)
• EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud Computing [9.41257807502252]
  We propose a novel paradigm named Encrypted Vision Transformer (EViT), which advances the discriminative representations capability of cipher-images. EViT achieves both excellent encryption and retrieval performance, outperforming current schemes in terms of retrieval accuracy by large margins while protecting image privacy effectively.
  arXiv  Detail & Related papers  (2022-08-31T07:07:21Z)
• Privacy-Preserving Image Classification Using ConvMixer with Adaptive Permutation Matrix [13.890279045382623]
  We propose a privacy-preserving image classification method using encrypted images under the use of the ConvMixer structure. Images with a large size cannot be applied to the conventional method with an adaptation network. We propose a novel method, which allows us not only to apply block-wise scrambled images to ConvMixer for both training and testing without the adaptation network.
  arXiv  Detail & Related papers  (2022-08-04T09:55:31Z)
• MaskGIT: Masked Generative Image Transformer [49.074967597485475]
  MaskGIT learns to predict randomly masked tokens by attending to tokens in all directions. Experiments demonstrate that MaskGIT significantly outperforms the state-of-the-art transformer model on the ImageNet dataset.
  arXiv  Detail & Related papers  (2022-02-08T23:54:06Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.