Spyre-Accelerated Retrieval-Augmented Generation on IBM LinuxONE: A Cloud-Native Architecture for Secure, High-Throughput Enterprise AI Inference

• URL: http://arxiv.org/abs/2608.21393v1
• Date: Sun, 02 Aug 2026 09:00:24 GMT
• Title: Spyre-Accelerated Retrieval-Augmented Generation on IBM LinuxONE: A Cloud-Native Architecture for Secure, High-Throughput Enterprise AI Inference
• Abstract summary: This paper lays out a six-subsystem RAG architecture that runs entirely on IBM LinuxONE.<n>Every piece of the pipeline from query through vector retrieval stays within a single LinuxONE system.<n>Early analysis points to end-to-end RAG latencies under two seconds and up to a 20x reduction compared to off-premises inference.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Running large language models inside enterprise environments has always bumped up against a practical wall: the data lives in one place, the AI horsepower sits somewhere else, and moving sensitive records between the two creates real headaches around latency, security, and regulatory exposure. IBM's Spyre accelerator PCIe inference card built for LinuxONE and the broader IBM Z family changes that equation. In this paper we lay out a six-subsystem RAG architecture that runs entirely on IBM LinuxONE, using Spyre for generative inference, the Telum II on-chip accelerator for lightweight classification tasks, and Red Hat OpenShift for container orchestration. Every piece of the pipeline from query intake through vector retrieval, prompt assembly, LLM inference, compliance filtering, and response delivery stays within a single LinuxONE system, so sensitive data never has to leave the hardware perimeter. We walk through the design choices behind each subsystem, dig into the Spyre compilation and serving stack, explain how LinuxONE's Secure Execution technology extends confidential-computing guarantees to AI workloads, and benchmark the architecture against cloud-GPU and on-premises alternatives. Early analysis points to end-to-end RAG latencies under two seconds and up to a 20x reduction compared to off-platform inference, all while keeping the strong encryption and auditability posture that regulated industries actually need.

Related papers

• HazardAuditor: From Executable Threats to Safer Computer-Use Agents [50.06351661912873]
  Existing guard models target static prompts and responses.<n>Existing safety platforms produce evaluation verdicts rather than the normalized supervision a guard model needs to learn.<n>We introduce HazardAuditor, an execution-grounded framework that closes both gaps.
  arXiv  Detail & Related papers  (2026-09-14T07:09:33Z)
• EnclaveX: End-to-End Confidential AI with CPU/GPU TEEs [0.26249027950824505]
  Large Language Models (LLMs) have rapidly proliferated, driving widespread adoption of AI applications.<n>Most deployments rely on centralized infrastructures such as Microsoft Azure, Google Cloud, or AWS, requiring users to share sensitive data and training or fine-tuning code.<n>This dependence raises significant security and privacy concerns, as cloud providers must be trusted to ensure confidentiality and integrity.<n>We propose mechanisms to ensure confidentiality and integrity at both the VM level (via Intel TDX and AMD SEV-SNP) and the application level, highlighting vulnerabilities such as administrators' ability to access confidential VM contents.
  arXiv  Detail & Related papers  (2026-06-30T09:35:28Z)
• Step-level Optimization for Efficient Computer-use Agents [51.29573359027217]
  We argue that strong computer-use agents remain expensive and slow in practice.<n>We propose an event-driven, step-level cascade for computer-use agents.
  arXiv  Detail & Related papers  (2026-04-29T19:59:36Z)
• Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt: A Machine Learning Approach [0.510691253204425]
  Ransomware core capability, unauthorized encryption, demands controls that identify and block malicious cryptographic activity without disrupting legitimate use.<n>We present a probabilistic, risk-based access control architecture that couples machine learning inference with mandatory access control to regulate encryption on Linux in real time.
  arXiv  Detail & Related papers  (2026-01-23T14:48:35Z)
• TZ-LLM: Protecting On-Device Large Language Models with Arm TrustZone [8.538298365840877]
  Large Language Models (LLMs) deployed on mobile devices offer benefits like user privacy and reduced network latency, but introduce a significant security risk.<n>We propose a system design for protecting on-device LLMs using Arm Trusted Execution Environment (TEE), TrustZone.
  arXiv  Detail & Related papers  (2025-11-17T18:59:20Z)
• CryptOracle: A Modular Framework to Characterize Fully Homomorphic Encryption [3.5348336893819554]
  This paper presents a detailed characterization of OpenFHE, a comprehensive open-source library for FHE.<n>We introduce CryptOracle, a modular evaluation framework comprising (1) a benchmark suite, (2) a hardware profiler, and (3) a predictive performance model.
  arXiv  Detail & Related papers  (2025-10-03T23:30:29Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• vApps: Verifiable Applications at Internet Scale [2.931173822616461]
  Verifiable Applications (vApps) is a novel development framework designed to streamline the creation and deployment of verifiable computing applications.<n>vApps offer a unified Rust-based Domain-Specific Language ( DSL) within a comprehensive SDK.<n>This eases the developer's burden in securing diverse software components, allowing them to focus on application logic.
  arXiv  Detail & Related papers  (2025-04-21T02:19:06Z)
• L2MAC: Large Language Model Automatic Computer for Extensive Code Generation [52.81694565226513]
  Transformer-based large language models (LLMs) are constrained by the fixed context window of the underlying transformer architecture.<n>This paper presents L2MAC, the first practical LLM-based general-purpose stored-program automatic computer (von Neumann architecture) framework, for long and consistent output generation.
  arXiv  Detail & Related papers  (2023-10-02T16:55:19Z)
• FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs [57.12856172329322]
  We envision a decentralized system unlocking the potential vast untapped consumer-level GPU. This system faces critical challenges, including limited CPU and GPU memory, low network bandwidth, the variability of peer and device heterogeneity.
  arXiv  Detail & Related papers  (2023-09-03T13:27:56Z)
• Monitoring ROS2: from Requirements to Autonomous Robots [58.720142291102135]
  This paper provides an overview of a formal approach to generating runtime monitors for autonomous robots from requirements written in a structured natural language. Our approach integrates the Formal Requirement Elicitation Tool (FRET) with Copilot, a runtime verification framework, through the Ogma integration tool.
  arXiv  Detail & Related papers  (2022-09-28T12:19:13Z)
• Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs [64.26714148634228]
  congestion control (CC) algorithms become extremely difficult to design. It is currently not possible to deploy AI models on network devices due to their limited computational capabilities. We build a computationally-light solution based on a recent reinforcement learning CC algorithm.
  arXiv  Detail & Related papers  (2022-07-05T20:42:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.