FEnc$^2$: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment Encoding

• URL: http://arxiv.org/abs/2606.16359v1
• Date: Mon, 15 Jun 2026 07:55:55 GMT
• Title: FEnc$^2$: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment Encoding
• Abstract summary: Homomorphic Encryption (FHE) enables privacy-preserving machine learning but incurs extreme computational and memory overhead.<n>FEnc2 is a unified and principled fragment-based encoding framework for CKKS-based private convolutional neural network inference.<n>It reduces ciphertext count and workload pressure on hardware, primitive-level optimizations such as NTT and keyswitch.
• Score: 26.667496203770074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fully Homomorphic Encryption (FHE) enables privacy-preserving machine learning but incurs extreme computational and memory overhead. These costs come not only from expensive low-level primitives, including Number Theoretic Transform (NTT), rotation, and key-switching, but also from inefficient ciphertext packing at the application level. Existing packing strategies typically preserve either neighboring data elements or feature grouping, but not both, leading to wasted ciphertext slots, excessive rotations, and inflated ciphertext counts. We propose FEnc2, a unified and principled fragment-based encoding framework for CKKS-based private convolutional neural network inference. FEnc2 optimizes slot utilization, rotation complexity, and ciphertext density through two components: 1)Conv-aware Encoding, which analytically selects an optimal fragment size to decouple spatial dependencies and jointly minimize inner-outer rotations across layers, and 2)Arch-aware Ct Compression, which restores ciphertext density after feature- or channel-reduction layers. Together, these transformations reshape encrypted workload structure and reduce homomorphic operations by one to two orders of magnitude. With full memory capacity utilized, i.e., at maximum batch size, FEnc2 achieves end-to-end latency speedups over the state-of-the-art Orion of up to 228.83x on GPU and 226.06x on CPU for LeNet on MNIST, and up to 4.55x on GPU and 9.43x on CPU for MobileNet on ImageNet. FEnc2 is hardware-agnostic yet architecturally transformative: by optimizing encrypted tensor layout before execution, it reduces ciphertext count and workload pressure on hardware, complementing primitive-level optimizations such as NTT and keyswitch accelerators. These results show that application-level data layout is a first-order architectural design dimension for encrypted inference and an important enabler for next-generation FHE systems.

Related papers

• Memory-Efficient Designs for Word-Wise Universal Fully Homomorphic Encryption [13.384404337894575]
  Homomorphic Encryption (FHE) enables computation on encrypted data, preserving privacy throughout analysis.<n>We propose BXT, an FHE optimization framework that mitigates the memory bottleneck through four techniques.<n>On CNN inference, the BXT-CSO50 configuration effectively achieves up to 3.8$times$ speedup over the 100x GPU baseline with less than 1% accuracy loss at 50% comparison precision.
  arXiv  Detail & Related papers  (2026-09-04T06:03:44Z)
• Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding [51.48425605758328]
  We present Faster Flash Decoding (FFD), a novel hardware-algorithm co-design framework designed to break the memory wall in long-context decoding.<n>FFD integrates the selector and computer into a fully fused kernel, replacing external metadata indices with content-aware scanning.<n>It achieves up to 11.6x kernel-level speedup and scaling to 256K context length, with 2.37x end-to-end throughput improvement.
  arXiv  Detail & Related papers  (2026-08-31T15:13:20Z)
• WHET: Welding Homomorphic Encryption to Accelerator Architectures [3.4027459785553]
  homomorphic encryption (FHE) enables computations on encrypted data without decryption.<n>Prior efforts have steadily improved FHE performance through cryptographic and algorithmic enhancements or hardware acceleration.<n>We introduce memory-centric, architecture-aware optimizations to better align cryptographic and algorithmic constructions with FHE accelerator architectures.
  arXiv  Detail & Related papers  (2026-06-10T01:04:58Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• Triple-Hoisted Baby-Step Giant-Step Linear Transformation over CKKS Homomorphic Encryption and Hardware Accelerator [3.050919759387984]
  homomorphic encryption (HE) is indispensable for privacy-preserving cloud computing.<n> Linear transformation is widely used in neural networks, including large language models.<n>This paper proposes a triple-hoisted baby-step giant-step algorithm that decomposes the baby step further to substantially reduce the number of ciphertext rotations needed for the CKKS HE evaluation.
  arXiv  Detail & Related papers  (2026-05-17T01:57:35Z)
• Fast SAM2 with Text-Driven Token Pruning [52.8350457627401]
  Segment Anything Model 2 (SAM2), a vision computation model has significantly advanced in prompt-driven video object segmentation.<n>SAM2 pipelines propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object.<n>We introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation.
  arXiv  Detail & Related papers  (2025-12-24T18:59:05Z)
• FedBit: Accelerating Privacy-Preserving Federated Learning via Bit-Interleaved Packing and Cross-Layer Co-Design [2.255961793913651]
  Federated learning (FL) with fully homomorphic encryption (FHE) effectively safeguards data privacy during model aggregation.<n>FedBit is a hardware/software co-designed framework for the Brakerski-Fan-Vercauteren (BFV) scheme.<n>FedBit employs bit-interleaved data packing to embed multiple model parameters into a single ciphertext coefficient.
  arXiv  Detail & Related papers  (2025-09-27T03:58:16Z)
• Re-Densification Meets Cross-Scale Propagation: Real-Time Neural Compression of LiDAR Point Clouds [83.39320394656855]
  LiDAR point clouds are fundamental to various applications, yet high-precision scans incur substantial storage and transmission overhead.<n>Existing methods typically convert unordered points into hierarchical octree or voxel structures for dense-to-sparse predictive coding.<n>Our framework comprises two lightweight modules. First, the Geometry Re-Densification Module re-densifies encoded sparse geometry, extracts features at denser scale, and then re-sparsifies the features for predictive coding.
  arXiv  Detail & Related papers  (2025-08-28T06:36:10Z)
• Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures [2.613335121517245]
  Fully homomorphic encryption (FHE) frees cloud computing from privacy concerns by enabling secure computation on encrypted data.<n>We present Cheddar, a high-performance FHE library for GPU, achieving substantial speedups over previous GPU implementations.
  arXiv  Detail & Related papers  (2024-07-17T23:49:18Z)
• INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing [66.00729477511219]
  Given a function represented as a computation graph, traditional architectures face challenges in efficiently computing its nth-order gradient. We introduce INR-Arch, a framework that transforms the computation graph of an nth-order gradient into a hardware-optimized dataflow architecture. We present results that demonstrate 1.8-4.8x and 1.5-3.6x speedup compared to CPU and GPU baselines respectively.
  arXiv  Detail & Related papers  (2023-08-11T04:24:39Z)
• BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance [54.214426436283134]
  Deep neural networks, such as the Deep-FSMN, have been widely studied for keyword spotting (KWS) applications. We present a strong yet efficient binary neural network for KWS, namely BiFSMNv2, pushing it to the real-network accuracy performance. We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25.1x speedup and 20.2x storage-saving on edge hardware.
  arXiv  Detail & Related papers  (2022-11-13T18:31:45Z)
• FFConv: Fast Factorized Neural Network Inference on Encrypted Data [9.868787266501036]
  We propose a low-rank factorization method called FFConv to unify convolution and ciphertext packing. Compared to prior art LoLa and Falcon, our method reduces the inference latency by up to 87% and 12%, respectively.
  arXiv  Detail & Related papers  (2021-02-06T03:10:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.