Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs

• URL: http://arxiv.org/abs/2608.26575v2
• Date: Tue, 01 Sep 2026 01:17:55 GMT
• Title: Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs
• Abstract summary: This paper measures the performance impact of running large language model inference and training inside a Trusted Execution Environment.<n>The performance impact is derived from paired confidential versus non-confidential runs on a single physical host.<n>We localize each cost to a specific encrypted boundary, give a microbenchmark that predicts the serving penalty to within a submission count, and end with concrete deployment guidance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper measures the performance impact of running large language model inference and training inside a Trusted Execution Environment (TEE) on NVIDIA B200 GPUs, using Intel Trust Domain Extensions (TDX) confidential VMs together with NVIDIA Confidential Computing (CC) on Blackwell GPUs. The performance impact is derived from paired confidential versus non-confidential runs on a single physical host where the only variable is the GPU CC bit and the TDX guest object in the VM launch. The main result is that confidential inference on Blackwell achieves low single-digit throughput overhead when the stack is configured correctly, at about 1-3%. Stock inference stacks incur 30 to 40% penalties due to avoidable configurations rather than the achievable operating point. The cost is not fully represented by a single number because it is governed by two independent axes, a fixed per-host-operation cost that amortizes as batch size grows and a per-NVLink-traffic cost that tracks the share of the step spent in encrypted collectives, and which of the two dominates is set by the workload and the software. We localize each cost to a specific encrypted boundary, give a microbenchmark that predicts the serving penalty to within a submission count, and end with concrete deployment guidance. GPU compute, energy draw, and usable memory capacity are unaffected by CC.

Related papers

• Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models [67.17268530365189]
  This paper showcases a memory-efficient training stack for Mixture-of-Experts (MoE) models.<n>It combines and specializes various existing and novel parallelism techniques at different layers and stages of the MoE model training pipeline.
  arXiv  Detail & Related papers  (2026-07-02T08:06:57Z)
• The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing [6.69555823710995]
  GPU-CC now preserves GPU-local performance on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance.<n>Yet serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double.<n>Bridge model explains a +131% KV-restore penalty and a 34x model slowdown.
  arXiv  Detail & Related papers  (2026-06-22T21:48:53Z)
• Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment [60.442064966340524]
  We present a prototype study on NVIDIA Jetson Orin Nano devices connected over WiFi.<n>Key finding is that the dominant bottleneck is not just network bandwidth but also the CPU- GPU staging during communication.<n>Experiments show that this strategy reduces latency by 65%-77% and energy consumption by 34%-52% relative to full-tensor exchange.
  arXiv  Detail & Related papers  (2026-05-25T10:39:28Z)
• Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs [2.2964417137558457]
  NVIDIA's CuTile introduces a Python-based, tile-centric abstraction for GPU kernel development.<n>We present the first independent, cross-architecture evaluation of CuTile against established approaches.
  arXiv  Detail & Related papers  (2026-04-25T23:13:47Z)
• A Scalable Multi-GPU Framework for Encrypted Large-Model Inference [5.966282323502589]
  Encrypted AI using fully homomorphic encryption (FHE) provides strong privacy guarantees.<n>Recent works proposed ASICs to accelerate FHE, but require expensive advanced manufacturing processes that constrain their computation.<n>This paper presents Cerium, a multi- GPU framework for FHE inference on large models.
  arXiv  Detail & Related papers  (2025-12-12T04:15:38Z)
• CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking [43.89894227481455]
  3D single object tracking in LiDAR point clouds is a critical task in computer vision and autonomous driving.<n>Despite great success having been achieved, the inherent sparsity of point clouds introduces a dual-redundancy challenge.<n>We propose CompTrack, a novel end-to-end framework that systematically eliminates both forms of redundancy in point clouds.
  arXiv  Detail & Related papers  (2025-11-19T16:12:24Z)
• Scalable GPU-Based Integrity Verification for Large Machine Learning Models [4.301162531343759]
  We present a security framework that strengthens distributed machine learning by standardizing integrity protections across CPU and GPU platforms.<n>Our approach co-locates integrity verification directly with large ML model execution on GPU accelerators.<n>We provide a hardware-agnostic foundation that enterprise teams can deploy regardless of their underlying CPU and GPU infrastructures.
  arXiv  Detail & Related papers  (2025-10-27T23:45:21Z)
• ThunderKittens: Simple, Fast, and Adorable AI Kernels [43.32681787348603]
  We present ThunderKittens (TK), a framework for writing performant AI kernels while remaining easy to use and maintain. We show the value of TK by providing kernels that match or outperform prior kernels for a range of AI operations.
  arXiv  Detail & Related papers  (2024-10-27T10:07:16Z)
• MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models [58.3342517278868]
  This paper describes the design of Mixed-precision AutoRegressive LINear kernels. It shows that batchsizes up to 16-32 can be supported with close to maximum ($4times$) quantization speedup. MarLIN accomplishes this via a combination of techniques, such as asynchronous memory access, complex task scheduling and pipelining.
  arXiv  Detail & Related papers  (2024-08-21T16:10:41Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption [33.87964584665433]
  Homomorphic Encryption (FHE) enables the processing of encrypted data without decrypting it. FHE introduces a slowdown of up to five orders of magnitude as compared to the same computation using plaintext data. We propose GME, which combines three key microarchitectural extensions along with a compile-time optimization to the current AMD CDNA GPU architecture.
  arXiv  Detail & Related papers  (2023-09-20T01:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.