GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption

• URL: http://arxiv.org/abs/2309.11001v1
• Date: Wed, 20 Sep 2023 01:50:43 GMT
• Title: GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption
• Abstract summary: Homomorphic Encryption (FHE) enables the processing of encrypted data without decrypting it. FHE introduces a slowdown of up to five orders of magnitude as compared to the same computation using plaintext data. We propose GME, which combines three key microarchitectural extensions along with a compile-time optimization to the current AMD CDNA GPU architecture.
• Score: 33.87964584665433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fully Homomorphic Encryption (FHE) enables the processing of encrypted data without decrypting it. FHE has garnered significant attention over the past decade as it supports secure outsourcing of data processing to remote cloud services. Despite its promise of strong data privacy and security guarantees, FHE introduces a slowdown of up to five orders of magnitude as compared to the same computation using plaintext data. This overhead is presently a major barrier to the commercial adoption of FHE. In this work, we leverage GPUs to accelerate FHE, capitalizing on a well-established GPU ecosystem available in the cloud. We propose GME, which combines three key microarchitectural extensions along with a compile-time optimization to the current AMD CDNA GPU architecture. First, GME integrates a lightweight on-chip compute unit (CU)-side hierarchical interconnect to retain ciphertext in cache across FHE kernels, thus eliminating redundant memory transactions. Second, to tackle compute bottlenecks, GME introduces special MOD-units that provide native custom hardware support for modular reduction operations, one of the most commonly executed sets of operations in FHE. Third, by integrating the MOD-unit with our novel pipelined $64$-bit integer arithmetic cores (WMAC-units), GME further accelerates FHE workloads by $19\%$. Finally, we propose a Locality-Aware Block Scheduler (LABS) that exploits the temporal locality available in FHE primitive blocks. Incorporating these microarchitectural features and compiler optimizations, we create a synergistic approach achieving average speedups of $796\times$, $14.2\times$, and $2.3\times$ over Intel Xeon CPU, NVIDIA V100 GPU, and Xilinx FPGA implementations, respectively.

Related papers

• Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding [51.48425605758328]
  We present Faster Flash Decoding (FFD), a novel hardware-algorithm co-design framework designed to break the memory wall in long-context decoding.<n>FFD integrates the selector and computer into a fully fused kernel, replacing external metadata indices with content-aware scanning.<n>It achieves up to 11.6x kernel-level speedup and scaling to 256K context length, with 2.37x end-to-end throughput improvement.
  arXiv  Detail & Related papers  (2026-08-31T15:13:20Z)
• Xe-Forge: Multi-Stage LLM-Powered Kernel Optimization for Intel GPU [38.81440160993858]
  Porting deep learning algorithms to new hardware accelerators requires developers to apply the same low-level optimizations to every Triton kernel in their code-base.<n>We present Xe-Forge, a multi-stage LLM-powered pipeline that automates this process for Intel GPU.<n>A curated knowledge base encodes Intel GPU constraints that are absent from LLM training data, keeping the model within architecturally valid bounds.
  arXiv  Detail & Related papers  (2026-04-16T20:21:01Z)
• Understand and Accelerate Memory Processing Pipeline for Disaggregated LLM Inference [45.78442013449325]
  Large language models rely on efficient long-context processing and generation mechanisms to support complex reasoning.<n>We show that these optimizations can be unified into a four-step memory processing pipeline: Prepare Memory, Compute Relevancy, Retrieval, and Apply to Inference.<n>Motivated by this insight, we argue that textbfheterogeneous systems are well-suited to accelerate memory processing and thus end-to-end inference.
  arXiv  Detail & Related papers  (2026-03-30T21:03:39Z)
• FHECore: Rethinking GPU Microarchitecture for Fully Homomorphic Encryption [2.7777199166440827]
  Fully Homomorphic Encryption (FHE) enables computation directly on encrypted data but incurs massive computational and memory overheads.<n>Custom accelerators can mitigate these costs, but their long time-to-market and the rapid evolution of FHE algorithms threaten their long-term relevance.<n>We propose FHECore, a specialized functional unit integrated directly into the GPU's Streaming Multiprocessor.
  arXiv  Detail & Related papers  (2026-02-10T02:55:10Z)
• Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple [42.09057806159106]
  General Matrix multiplication is the cornerstone of Deep Learning and HPC workloads.<n>Modern platforms with matrix multiplication accelerators exhibit high FLOP/Byte machine balance.<n>In this work we revisit space filling curves (SFC) to alleviate the problem of this cumbersome tuning.<n>We obtain platform-oblivious and shape-oblivious matrix-multiplication schemes that exhibit inherently high degree of data locality.
  arXiv  Detail & Related papers  (2026-01-22T19:56:16Z)
• Theodosian: A Deep Dive into Memory-Hierarchy-Centric FHE Acceleration [3.8153115302044296]
  Fully homomorphic encryption (FHE) enables secure computation on encrypted data, mitigating privacy concerns in cloud and edge environments.<n>In this paper, we perform a microarchitectural analysis of CKKS, a popular FHE scheme, on modern GPUs.<n>We show that the dominant kernels remain bound by memory bandwidth despite a high-bandwidth L2 cache, exposing a persistent memory wall.<n>Motivated by these findings, we introduce Theodosian, a set of complementary, memory-aware optimizations that improve cache efficiency and reduce overheads.
  arXiv  Detail & Related papers  (2025-12-20T12:18:29Z)
• ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels [40.94392896555992]
  Existing systems mitigate this through compute-communication overlap but often fail to meet theoretical bandwidth across workloads and new accelerators.<n>Instead of operator-specific techniques, we ask whether a small set of simple, reusable principles can guide the optimal optimal performance of workloads.<n>PKKittens (PK) kernels achieves up to $2.33 times times parallel workloads.
  arXiv  Detail & Related papers  (2025-11-17T21:48:33Z)
• CryptOracle: A Modular Framework to Characterize Fully Homomorphic Encryption [3.5348336893819554]
  This paper presents a detailed characterization of OpenFHE, a comprehensive open-source library for FHE.<n>We introduce CryptOracle, a modular evaluation framework comprising (1) a benchmark suite, (2) a hardware profiler, and (3) a predictive performance model.
  arXiv  Detail & Related papers  (2025-10-03T23:30:29Z)
• CAT: A GPU-Accelerated FHE Framework with Its Application to High-Precision Private Dataset Query [0.51795041186793]
  We introduce an open-source GPU-accelerated fully homomorphic encryption (FHE) framework CAT. emphCAT features a three-layer architecture: a foundation of core math, a bridge of pre-computed elements and combined operations, and an API-accessible layer of FHE operators. Based on our framework, we implement three widely used FHE schemes: CKKS, BFV, and BGV.
  arXiv  Detail & Related papers  (2025-03-28T08:20:18Z)
• gECC: A GPU-based high-throughput framework for Elliptic Curve Cryptography [15.39096542261856]
  Elliptic Curve Cryptography (ECC) is an encryption method that provides security comparable to traditional techniques like Rivest-Shamir-Adleman (RSA) ECC is still hindered by the significant performance overhead associated with elliptic curve (EC) operations. This paper presents gECC, a versatile framework for ECC optimized for GPU architectures.
  arXiv  Detail & Related papers  (2024-12-22T01:50:50Z)
• Chameleon: An Efficient FHE Scheme Switching Acceleration on GPUs [17.536473118470774]
  homomorphic encryption (FHE) enables direct computation on encrypted data. Existing efforts primarily focus on single-class FHE schemes, which fail to meet the diverse requirements of data types and functions. We present an efficient GPU-based FHE switching acceleration scheme named Chameleon.
  arXiv  Detail & Related papers  (2024-10-08T11:37:49Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• Cheddar: A Swift Fully Homomorphic Encryption Library for CUDA GPUs [2.613335121517245]
  Fully homomorphic encryption (FHE) is a cryptographic technology capable of resolving security and privacy problems in cloud computing by encrypting data in use. FHE introduces tremendous computational overhead for processing encrypted data, causing FHE workloads to become 2-6 orders of magnitude slower than their unencrypted counterparts. We propose Cheddar, an FHE library for GPU, which demonstrates significantly faster performance compared to prior GPU implementations.
  arXiv  Detail & Related papers  (2024-07-17T23:49:18Z)
• MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter [40.616849959987555]
  We introduce a novel mechanism that fine-tunes Large Language Models (LLMs) with adapters of larger size yet memory-efficient. This is achieved by leveraging the inherent activation sparsity in the Feed-Forward Networks (FFNs) of LLMs. We employ a Mixture of Experts (MoE)-like architecture to mitigate unnecessary CPU computations and reduce the communication volume between the GPU and CPU.
  arXiv  Detail & Related papers  (2024-06-07T14:49:22Z)
• NTTSuite: Number Theoretic Transform Benchmarks for Accelerating Encrypted Computation [2.704681057324485]
  Homomorphic encryption (HE) is a cryptographic system that enables computation directly on encrypted data. HE has seen little adoption due to extremely high computational overheads, rendering it impractical. We develop a benchmark suite, named NTTSuite, to enable researchers to better address these overheads. We find our implementation outperforms the state-of-the-art by 30%.
  arXiv  Detail & Related papers  (2024-05-18T17:44:17Z)
• FHEmem: A Processing In-Memory Accelerator for Fully Homomorphic Encryption [9.884698447131374]
  Homomorphic Encryption (FHE) is a technique that allows arbitrary computations to be performed on encrypted data without the need for decryption. FHE is significantly slower than computation on plain data due to the increase in data size after encryption. We propose a PIM-based FHE accelerator, FHEmem, which exploits a novel processing in-memory architecture.
  arXiv  Detail & Related papers  (2023-11-27T20:11:38Z)
• Toward Practical Privacy-Preserving Convolutional Neural Networks Exploiting Fully Homomorphic Encryption [11.706881389387242]
  Homomorphic encryption (FHE) is a viable approach for achieving private inference (PI) FHE implementation of a CNN faces significant hurdles, primarily due to FHE's substantial computational and memory overhead. We propose a set of optimizations, which includes GPU/ASIC acceleration, an efficient activation function, and an optimized packing scheme.
  arXiv  Detail & Related papers  (2023-10-25T10:24:35Z)
• FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs [57.12856172329322]
  We envision a decentralized system unlocking the potential vast untapped consumer-level GPU. This system faces critical challenges, including limited CPU and GPU memory, low network bandwidth, the variability of peer and device heterogeneity.
  arXiv  Detail & Related papers  (2023-09-03T13:27:56Z)
• INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing [66.00729477511219]
  Given a function represented as a computation graph, traditional architectures face challenges in efficiently computing its nth-order gradient. We introduce INR-Arch, a framework that transforms the computation graph of an nth-order gradient into a hardware-optimized dataflow architecture. We present results that demonstrate 1.8-4.8x and 1.5-3.6x speedup compared to CPU and GPU baselines respectively.
  arXiv  Detail & Related papers  (2023-08-11T04:24:39Z)
• SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers [76.13755422671822]
  This paper investigates the capability of plain Vision Transformers (ViTs) for semantic segmentation using the encoder-decoder framework. We introduce a novel Attention-to-Mask (atm) module to design a lightweight decoder effective for plain ViT. Our decoder outperforms the popular decoder UPerNet using various ViT backbones while consuming only about $5%$ of the computational cost.
  arXiv  Detail & Related papers  (2023-06-09T22:29:56Z)
• ASH: A Modern Framework for Parallel Spatial Hashing in 3D Perception [91.24236600199542]
  ASH is a modern and high-performance framework for parallel spatial hashing on GPU. ASH achieves higher performance, supports richer functionality, and requires fewer lines of code. ASH and its example applications are open sourced in Open3D.
  arXiv  Detail & Related papers  (2021-10-01T16:25:40Z)
• Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
  Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field. We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem. We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
  arXiv  Detail & Related papers  (2020-06-17T18:14:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.