Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX

• URL: http://arxiv.org/abs/2607.19353v1
• Date: Wed, 20 May 2026 22:52:12 GMT
• Title: Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX
• Abstract summary: This paper presents a benchmark study comparing standard non-confidential execution with confidential computing mode on a single NVIDIA H100 80GB GPU.<n>The results suggest that confidential GPU inference can retain usable throughput under load, but capacity planning must account for both the steady throughput penalty and the earlier saturation behavior observed for larger models.
• Score: 2.6853734738584047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary model assets. However, the performance cost of enabling confidential execution for GPU-accelerated large language model serving remains workload dependent and operationally important. This paper presents a benchmark study comparing standard non-confidential execution with confidential computing mode on a single NVIDIA H100 80GB GPU hosted in an Intel TDX confidential instance. The evaluation uses two representative language models, Mistral-7B v0.1 and Qwen3-30B-A3B, and measures time to first token, end-to-end request latency, per-request token generation throughput, global token throughput, and closed-loop request throughput under increasing concurrency. In fixed request-rate experiments, confidential mode increases average TTFT by 21.8% for Mistral-7B and 27.8% for Qwen3-30B-A3B, while global token throughput drops by 17.7% and 21.1%, respectively. In closed-loop concurrency experiments, throughput gaps remain in the 11.5-20.2% range, but the larger model reaches its saturation knee earlier under confidential mode. The results suggest that confidential GPU inference can retain usable throughput under load, but capacity planning must account for both the steady throughput penalty and the earlier saturation behavior observed for larger models.

Related papers

• Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs [0.0]
  This paper measures the performance impact of running large language model inference and training inside a Trusted Execution Environment.<n>The performance impact is derived from paired confidential versus non-confidential runs on a single physical host.<n>We localize each cost to a specific encrypted boundary, give a microbenchmark that predicts the serving penalty to within a submission count, and end with concrete deployment guidance.
  arXiv  Detail & Related papers  (2026-08-27T03:37:35Z)
• BoltNet: An Ultra-Lightweight Convolutional Network for On-Device Plant Species Identification [3.860597465886636]
  BoltNet is an ultra-lightweight fully convolutional architecture combining a Spatial Redistribution Bottleneck and Logit PreSampling.<n>On Pl@ntNet300K, BoltNet reaches 0.682 F1-score with 341K parameters (1.37 MB), the highest F1-score among evaluated models below 2 MB.<n>BoltNet is the most consistently efficient model, with the best FPS/W on the GPU and NPU and second-best on the CPU.
  arXiv  Detail & Related papers  (2026-08-12T09:34:52Z)
• Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening [6.384763560610076]
  We report operational experience full-fine-tuning a 32.76B- parameter dense model (Qwen3-32B) on 16 x NVIDIA B300 (two nodes, FSDP / ZeRO-3)<n>Our contribution is the integrated field experience and a set of calibrated measurements on new hardware.<n>We position our instantiation against PyTorch's documented Join / equalize-to-minimum practice.
  arXiv  Detail & Related papers  (2026-08-06T12:11:25Z)
• KernelSight-LM: A Kernel-Level LLM Inference Simulator [27.54401355597591]
  We present KernelSight-LM, a fine-grained inference simulator that models token-level execution and produces kernel-level latency breakdowns.<n>It decomposes each serving step into a kernel model with a learned efficiency term, a communication model, and a host-over-head model.<n>Its kernel-level bottleneck breakdowns support hardware/software co-design and capacity planning.
  arXiv  Detail & Related papers  (2026-06-26T19:43:38Z)
• KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch? [13.026983194416168]
  Large language models (LLMs) can generate custom kernels that appear to outperform PyTorch on benchmarks such as KernelBench.<n>We demonstrate that frontier models frequently engage in reward hacking to artificially inflate reported performance.
  arXiv  Detail & Related papers  (2026-06-26T03:22:50Z)
• The Model Parking Tax: Quantifying the Hidden Energy Cost of Always-On GPU Model Deployment [0.0]
  AI industry keeps models loaded in memory around the clock to avoid cold-start latency.<n>Yet the structure of this cost has never been empirically decomposed.<n>We present the first cross-architecture measurement of idle GPU power as a function of VRAM allocation.
  arXiv  Detail & Related papers  (2026-04-15T09:01:24Z)
• GPU-Accelerated Optimization of Transformer-Based Neural Networks for Real-Time Inference [0.0]
  This paper presents the design and evaluation of a GPU-accelerated inference pipeline for transformer models using NVIDIART with mixed-precision optimization.<n>The system achieves up to 64.4x speedup over CPU baselines, sub-10 ms latency for single-sample inference, and a 63 percent reduction in memory usage.
  arXiv  Detail & Related papers  (2026-03-30T17:27:33Z)
• Can We Predict Before Executing Machine Learning Agents? [74.39460101251792]
  We formalize the task of Data-centric Solution Preference and construct a comprehensive corpus of 18,438 pairwise comparisons.<n>We demonstrate that LLMs exhibit significant predictive capabilities when primed with a Verified Data Analysis Report.<n>We instantiate this framework in FOREAGENT, an agent that employs a Predict-then-Verify loop, achieving a 6x acceleration in convergence while surpassing execution-based baselines by +6%.
  arXiv  Detail & Related papers  (2026-01-09T16:44:17Z)
• dParallel: Learnable Parallel Decoding for dLLMs [77.24184219948337]
  Diffusion large language models (dLLMs) offer parallel token prediction and lower inference latency.<n>Existing open-source models still require nearly token-length decoding steps to ensure performance.<n>We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.
  arXiv  Detail & Related papers  (2025-09-30T16:32:52Z)
• Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction [95.91743732150233]
  Goedel-Prover-V2, a series of open-source language models, set a new state-of-the-art in automated theorem proving.<n>We generate synthetic tasks of increasing difficulty to train the model to master increasingly complex theorems.<n>Goedel-Prover-V2-32B achieves 88.1% on MiniF2F at pass@32 in standard mode and 90.4% in self-correction mode.
  arXiv  Detail & Related papers  (2025-08-05T16:28:22Z)
• 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float [52.079202872069835]
  Large-scale AI models, such as Large Language Models (LLMs) and Diffusion Models (DMs) have grown rapidly in size.<n>We introduce Dynamic-Length Float (DFloat11), a compression framework that reduces LLM and DM size by 30% while preserving outputs that are bit-for-bit identical to the original model.
  arXiv  Detail & Related papers  (2025-04-15T22:38:38Z)
• START: Self-taught Reasoner with Tools [51.38785489790888]
  We introduce START (Self-Taught Reasoner with Tools), a tool-integrated long Chain-of-thought (CoT) reasoning LLM.<n> START is capable of performing complex computations, self-checking, exploring diverse methods, and self-ging.<n>It significantly outperforms the base QwQ-32B and achieves performance comparable to the state-of-the-art open-weight model R1-Distill-Qwen-32B.
  arXiv  Detail & Related papers  (2025-03-06T17:11:51Z)
• DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables [49.965024476651706]
  DeepGEMM is a lookup table based approach for the execution of ultra low-precision convolutional neural networks on SIMD hardware. Our implementation outperforms corresponding 8-bit integer kernels by up to 1.74x on x86 platforms.
  arXiv  Detail & Related papers  (2023-04-18T15:13:10Z)
• ETAD: A Unified Framework for Efficient Temporal Action Detection [70.21104995731085]
  Untrimmed video understanding such as temporal action detection (TAD) often suffers from the pain of huge demand for computing resources. We build a unified framework for efficient end-to-end temporal action detection (ETAD) ETAD achieves state-of-the-art performance on both THUMOS-14 and ActivityNet-1.3.
  arXiv  Detail & Related papers  (2022-05-14T21:16:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.