Tessera: Secure, Near-Line-Rate Weight Streaming for UMA Edge Accelerators

• URL: http://arxiv.org/abs/2604.23205v1
• Date: Sat, 25 Apr 2026 08:29:50 GMT
• Title: Tessera: Secure, Near-Line-Rate Weight Streaming for UMA Edge Accelerators
• Abstract summary: Tessera is a reference architecture for inline, cache-line weight decryption on UMA edge accelerators.<n>The design intercepts 64-byte AXI bursts, computing AES-256-CTR keystreams in sparse parallel with DRAM fetches.<n>We show that Tessera is projected to achieve 98.4% of the theoretical memory bandwidth ceiling.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deploying proprietary Deep Neural Networks (DNNs) on commodity edge devices demands hardware-backed Digital Rights Management (DRM) capable of withstanding both software-level and physical adversaries. In Unified Memory Architecture (UMA) systems, the host CPU and Neural Processing Unit (NPU) share physical DRAM, leaving plaintext model weights directly readable by a compromised OS kernel. Existing defenses fail in this constrained setting: trusted execution environments monopolize scarce memory with permanently reserved regions, while full-memory encryption operates at page granularity. This forces the system to fetch massive 4 KB memory pages for sub-page tensor tiles, severely crippling bandwidth. We present Tessera, a reference architecture for inline, cache-line granularity weight decryption on UMA edge accelerators. The design intercepts 64-byte AXI bursts, computing AES-256-CTR keystreams in parallel with DRAM fetches. This streams plaintext directly into isolated NPU SRAM, creating a transient memory footprint confined to the active tile and eliminating the need for permanent memory carve-outs. Measurements across three distinct SoC platforms demonstrate that this parallelization hides cryptographic latency behind standard DRAM fetch times, a condition that holds even under worst-case timing variations. Consequently, Tessera is projected to achieve 98.4\% of the theoretical memory bandwidth ceiling (a mere 1.6\% overhead). Across standard vision and language models, page-level memory encryption suffers up to a 32x bandwidth penalty, whereas Tessera maintains an optimal 1x footprint for all layer geometries. Finally, Tessera neutralizes major UMA-specific attack vectors -- including physical DRAM extraction, rogue DMA, and compute hijacking -- and formally prevents plaintext leakage across sparse tensors.

Related papers

• Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding [51.48425605758328]
  We present Faster Flash Decoding (FFD), a novel hardware-algorithm co-design framework designed to break the memory wall in long-context decoding.<n>FFD integrates the selector and computer into a fully fused kernel, replacing external metadata indices with content-aware scanning.<n>It achieves up to 11.6x kernel-level speedup and scaling to 256K context length, with 2.37x end-to-end throughput improvement.
  arXiv  Detail & Related papers  (2026-08-31T15:13:20Z)
• CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving [2.408411453763233]
  Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration.<n>Co-tenant workloads can further access memory-mapped interfaces or RDMA regions without physical co-location.<n>We present CloakLM, a software-only memory-obfuscation framework that removes this regularity without changing the inference stack's logical view of memory.
  arXiv  Detail & Related papers  (2026-06-16T18:47:47Z)
• FEnc$^2$: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment Encoding [26.667496203770074]
  Homomorphic Encryption (FHE) enables privacy-preserving machine learning but incurs extreme computational and memory overhead.<n>FEnc2 is a unified and principled fragment-based encoding framework for CKKS-based private convolutional neural network inference.<n>It reduces ciphertext count and workload pressure on hardware, primitive-level optimizations such as NTT and keyswitch.
  arXiv  Detail & Related papers  (2026-06-15T07:55:55Z)
• FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention [77.12062766962815]
  Lookahead Sparse Attention (LSA) is powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture.<n>We instantiate this architecture via a backbone-free decoupled training strategy.<n>We show that FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline.
  arXiv  Detail & Related papers  (2026-06-08T06:25:54Z)
• Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts [68.79341332280062]
  Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time.<n>We introduce OOMB, a highly memory-efficient training system that directly confronts this barrier.<n>Our approach employs a chunk-recurrent training framework with on-the-fly activation recomputation, which maintains a constant activation memory footprint.
  arXiv  Detail & Related papers  (2026-02-02T13:52:40Z)
• Radiant Foam Rendering on a Graph Processor [25.181643235594734]
  Many-core accelerators replace a single large device memory with hundreds to thousands of lightweight cores, each owning only a small local and exchanging data via explicit on-chip communication.<n>This organization offers high aggregate bandwidth, but it breaks a key assumption behind many techniques: that rays can randomly access a large, unified scene representation.<n>We present a fully in rendering-SRAM, distributed for the Radiant Foam Voronoi-cell volumetric representation on the Graphcore Mk2 IPU.<n>Our system attains throughput of approximately 1 fps at 640x480 with image and depth map quality close to the original GPU-based Radiant Foam implementation
  arXiv  Detail & Related papers  (2026-01-07T20:44:04Z)
• Bare-Metal Tensor Virtualization: Overcoming the Memory Wall in Edge-AI Inference on ARM64 [0.5729426778193398]
  "Virtual Core" architecture implemented in software optimized for ARM64 microarchitectures (Apple Silicon)<n>"Software-Defined Direct Memory Access (DMA)" guarantees 100% cache line utilization for weight, while our zero-copy loader eliminates latency.<n> Experimental results on a 110M-second model demonstrate a stable throughput of >60 tokens/second on M2 hardware.
  arXiv  Detail & Related papers  (2026-01-06T15:00:40Z)
• Kelle: Co-design KV Caching and eDRAM for Efficient LLM Serving in Edge Computing [9.984481065465028]
  Large Language Models (LLMs) on edge devices are crucial for reducing latency, improving real-time processing, and enhancing privacy.<n> implementing LLMs on edge devices presents challenges, particularly with managing key-value caches.<n>We propose eDRAM as the primary storage for LLM serving in edge device, which offers higher density compared to storage.
  arXiv  Detail & Related papers  (2025-10-16T07:12:08Z)
• FedBit: Accelerating Privacy-Preserving Federated Learning via Bit-Interleaved Packing and Cross-Layer Co-Design [2.255961793913651]
  Federated learning (FL) with fully homomorphic encryption (FHE) effectively safeguards data privacy during model aggregation.<n>FedBit is a hardware/software co-designed framework for the Brakerski-Fan-Vercauteren (BFV) scheme.<n>FedBit employs bit-interleaved data packing to embed multiple model parameters into a single ciphertext coefficient.
  arXiv  Detail & Related papers  (2025-09-27T03:58:16Z)
• Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction [72.27673320976933]
  Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding.<n>Current caching techniques accelerate decoding by storing full-layer states, yet impose substantial memory usage.<n>We propose Sparse-dLLM, the first training-free framework integrating dynamic cache eviction with sparse attention.
  arXiv  Detail & Related papers  (2025-08-04T16:14:03Z)
• BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments [53.71158537264695]
  Large language models (LLMs) have revolutionized numerous applications, yet their deployment remains challenged by memory constraints on local devices.<n>We introduce textbfBitStack, a novel, training-free weight compression approach that enables megabyte-level trade-offs between memory usage and model performance.
  arXiv  Detail & Related papers  (2024-10-31T13:26:11Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• Enabling Homomorphically Encrypted Inference for Large DNN Models [1.0679692136113117]
  Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x--10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources. We explore the feasibility of leveraging hybrid memory systems comprised of DRAM and persistent memory.
  arXiv  Detail & Related papers  (2021-03-30T07:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.