OpenPCC: Open and Confidential LLM Serving on Commodity TEEs

• URL: http://arxiv.org/abs/2606.11145v1
• Date: Tue, 09 Jun 2026 17:33:33 GMT
• Title: OpenPCC: Open and Confidential LLM Serving on Commodity TEEs
• Abstract summary: We present our analysis of the fundamental requirements of building a secure yet open Cloud Inference Service.<n>We implement an open-source prototype and characterize it end-to-end on a Llama-3 8B vLLM workload.<n>Our analysis and evaluation demonstrated the feasibility and security of the system.
• Score: 9.964425648424017
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generative AI applications such as personal AI agents, image generators, and chat assistants offer advanced capabilities to improve user experience. Behind the scenes, Large Language Models (LLMs) that power these services require a massive amount of computation and are usually deployed in the cloud, available as APIs, meaning that a user's request has to be sent to a Cloud Inference Service (CIS) for processing. However, the strong capabilities of LLM also mean that user's requests now contain much more personal sensitive or enterprise confidential information, demanding equally strong protection in CIS. While early industry efforts such as Apple Private Cloud Compute (PCC) and Google Private AI Compute have emerged to show the potential of secure CIS, they are not adoptable for deployment by others due to their reliance on proprietary hardware and closed ecosystem. In addition, they all suffer from their own design glitches that can undermine the ambitious goal of bringing in true privacy protection to end users. In this paper, we present our analysis of the fundamental requirements of building a secure yet open CIS. We then present OpenPCC, a Confidential CIS framework that does not rely on proprietary hardware but instead uses commercially available TEEs. We implement an open-source prototype and characterize it end-to-end on a Llama-3 8B vLLM workload, separating OpenPCC's own cost from the underlying TEE hardware. Our analysis and evaluation demonstrated the feasibility and security of the system.

Related papers

• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• Decentralized Compute on Untrusted Hardware Using Intel TDX and Encrypted CVMs [0.0]
  This paper introduces a decentralized, confidential computing platform that leverages Intel Trust Domain Extensions (TDX), Intel Trust Authority (ITA) and NVIDIA Confidential Computing (CC)<n>The proposed architecture incentivizes hardware providers to contribute Intel TDX capable compute resources.<n>Each participating provider is provisioned with a freshly instantiated, uniquely encrypted Ubuntu 24.04 CVM, providing data protection across all stages, at rest, in transit, and in use.
  arXiv  Detail & Related papers  (2026-07-23T23:44:27Z)
• RedSage: A Cybersecurity Generalist LLM [45.91667919408369]
  RedSage is an open-source, locally deployable cybersecurity assistant with domain-aware pretraining and post-training.<n>We use a large-scale web filtering and manual collection of high-quality resources, spanning 28.6K documents across frameworks, offensive techniques, and security tools.<n>RedSage is evaluated on established cybersecurity benchmarks (e.g., CTI-Bench, CyberMetric, SECURE) and general LLM benchmarks to assess broader generalization.
  arXiv  Detail & Related papers  (2026-01-29T18:59:57Z)
• OpenCUA: Open Foundations for Computer-Use Agents [74.61449905487565]
  Vision-language models have demonstrated impressive capabilities as computer-use agents (CUAs)<n>We propose OpenCUA, a comprehensive open-source framework for scaling CUA data and foundation models.<n>Our end-to-end agent models demonstrate strong performance across CUA benchmarks.
  arXiv  Detail & Related papers  (2025-08-12T17:52:32Z)
• Privacy Preservation in Gen AI Applications [0.0]
  Large Language Models (LLMs) may unintentionally absorb and reveal Personally Identifiable Information (PII) from user interactions.<n>Deep neural networks' intricacy makes it difficult to track down or stop the inadvertent storing and release of private information.<n>This study tackles these issues by detecting Generative AI weaknesses through attacks such as data extraction, model inversion, and membership inference.<n>It ensures privacy without sacrificing functionality by using methods to identify, alter, or remove PII before to dealing with LLMs.
  arXiv  Detail & Related papers  (2025-04-12T06:19:37Z)
• Privacy-Preserving Decentralized AI with Confidential Computing [0.7893328752331561]
  This paper addresses privacy protection in decentralized Artificial Intelligence (AI) using Confidential Computing (CC) within the Atoma Network. CC leverages hardware-based Trusted Execution Environments (TEEs) to provide isolation for processing sensitive data. We explore how we can integrate TEEs into Atoma's decentralized framework.
  arXiv  Detail & Related papers  (2024-10-17T16:50:48Z)
• Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends [64.57762280003618]
  It is foreseeable that in the near future, LM-driven general AI agents will serve as essential tools in production tasks.<n>This paper investigates scenarios involving the autonomous collaboration of future LM agents.
  arXiv  Detail & Related papers  (2024-09-22T14:09:49Z)
• Confidential Federated Computations [16.415880530250092]
  Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data.<n>FLA systems do not necessarily require anonymization mechanisms like differential privacy (DP)<n>This paper introduces a novel system architecture that leverages trusted execution environments (TEEs) and open-sourcing to ensure confidentiality of server-side computations.
  arXiv  Detail & Related papers  (2024-04-16T17:47:27Z)
• HasTEE+ : Confidential Cloud Computing and Analytics with Haskell [50.994023665559496]
  Confidential computing enables the protection of confidential code and data in a co-tenanted cloud deployment using specialized hardware isolation units called Trusted Execution Environments (TEEs) TEEs offer low-level C/C++-based toolchains that are susceptible to inherent memory safety vulnerabilities and lack language constructs to monitor explicit and implicit information-flow leaks. We address the above with HasTEE+, a domain-specific language (cla) embedded in Haskell that enables programming TEEs in a high-level language with strong type-safety.
  arXiv  Detail & Related papers  (2024-01-17T00:56:23Z)
• The Security and Privacy of Mobile Edge Computing: An Artificial Intelligence Perspective [64.36680481458868]
  Mobile Edge Computing (MEC) is a new computing paradigm that enables cloud computing and information technology (IT) services to be delivered at the network's edge. This paper provides a survey of security and privacy in MEC from the perspective of Artificial Intelligence (AI) We focus on new security and privacy issues, as well as potential solutions from the viewpoints of AI.
  arXiv  Detail & Related papers  (2024-01-03T07:47:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.