CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs

• URL: http://arxiv.org/abs/2608.00720v1
• Date: Sat, 01 Aug 2026 15:40:13 GMT
• Title: CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs
• Abstract summary: We present CascadeLUT, an information-structured inference framework organized around bandwidth constraints.<n>By co-designing feature scheduling with hardware dataflow, CascadeLUT reduces data movement while maintaining accuracy.<n>Across datasets, it achieves 4.0 to 12.5 times lower latency, 3.0 to 5.0 times higher throughput and up to 13.8 times lower energy/sample.
• Score: 1.4312099612976772
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mapping neural networks to FPGAs enables low-latency, energy-efficient inference, particularly for lookup table (LUT)-based models that eliminate multipliers and map directly to reconfigurable fabric. While prior work achieves high compute efficiency, it typically assumes full-sample availability, causing pipeline stalls in bandwidth-limited streaming scenarios. Here, the bottleneck shifts from computation to data movement, as large input transfers limit throughput and energy efficiency. We present CascadeLUT, an information-structured inference framework organized around bandwidth constraints. Instead of buffering the full input, features are partitioned into ordered subsets and predictions are progressively refined as subsets arrive. The cascade statically controls which layers consume incoming features, enabling deterministic streaming inference without runtime branching. By co-designing feature scheduling with hardware dataflow, CascadeLUT reduces data movement while maintaining accuracy. Across datasets, it achieves 4.0 to 12.5 times lower latency, 3.0 to 5.0 times higher throughput and up to 13.8 times lower energy/sample than prior LUT baselines, using 1.2 to 4.4 times the LUTs of the smallest DWN baseline per task. We also demonstrate on-device input quantization integrated with LUT-based inference and present end-to-end FPGA results on real-world workloads, with 5 times reductions in quantization overhead.

Related papers

• ELiTeFormer: An Efficient Transformer for FPGAs [3.884894816711594]
  Transformer blocks are prevalent in large language model (LLM) but present deployment challenges.<n>We present ELiTeFormer, the first Transformer model architecture that unifies hybrid linear attention with ultra-low-precision (ternary) linear projections.
  arXiv  Detail & Related papers  (2026-07-04T00:52:06Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• QUILL: An Algorithm-Architecture Co-Design for Cache-Local Deformable Attention [12.542462936966844]
  QUILL is a schedule-aware accelerator that turns deformable attention into cache-friendly, single-pass work.<n>A fused MSDeformAttn engine executes, Softmax, aggregation, and the final projection (W''m) in one pass without spilling intermediates.
  arXiv  Detail & Related papers  (2025-11-17T18:34:04Z)
• TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAs [9.646882213709814]
  TeLLMe is a table-lookup-based ternary LLM accelerator for low-power edge FPGAs.<n>It supports both prefill and autoregressive decoding using 1.58-bit weights and 8-bit activations.<n>Under a 5W power budget, TeLLMe delivers up to 25tokens/s decoding throughput.
  arXiv  Detail & Related papers  (2025-10-03T05:37:51Z)
• da4ml: Distributed Arithmetic for Real-time Neural Networks on FPGAs [5.979741271992278]
  We propose an efficient algorithm for implementing constant matrix-vector multiplication (CMVM) operations with distributed arithmetic (DA) on FPGAs.<n>The algorithm achieves resource reduction similar to state-of-the-art algorithms while being significantly faster to compute.<n>We show that the proposed algorithm can reduce on-chip resources by up to a third for realistic, highly quantized neural networks while simultaneously reducing latency.
  arXiv  Detail & Related papers  (2025-07-06T21:01:32Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• Fast Matrix Multiplications for Lookup Table-Quantized LLMs [58.11584672945781]
  FLUTE is a flexible lookup table engine for LUT-quantized LLMs.<n>At batch sizes 32 and quantization group size of 128, the FLUTE kernel can be 2-4x faster than existing GEMM kernels.
  arXiv  Detail & Related papers  (2024-07-15T17:55:42Z)
• NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions [2.7086888205833968]
  Field-Programmable Gate Array (FPGA) accelerators have proven successful in handling latency- and resource-critical deep neural network (DNN) inference tasks. We propose relaxing the boundaries of neurons and mapping entire sub-networks to a single LUT. We validate our proposed method on a known latency-critical task, jet substructure tagging, and on the classical computer vision task, digit classification using MNIST.
  arXiv  Detail & Related papers  (2024-02-29T16:10:21Z)
• Federated Learning for Energy-limited Wireless Networks: A Partial Model Aggregation Approach [79.59560136273917]
  limited communication resources, bandwidth and energy, and data heterogeneity across devices are main bottlenecks for federated learning (FL) We first devise a novel FL framework with partial model aggregation (PMA) The proposed PMA-FL improves 2.72% and 11.6% accuracy on two typical heterogeneous datasets.
  arXiv  Detail & Related papers  (2022-04-20T19:09:52Z)
• Network Calculus with Flow Prolongation -- A Feedforward FIFO Analysis enabled by ML [73.11023209243326]
  Flow Prolongation (FP) has been shown to improve delay bound accuracy significantly. We introduce DeepFP, an approach to make FP scale by predicting prolongations using machine learning. DeepFP reduces delay bounds by 12.1% on average at negligible additional computational cost.
  arXiv  Detail & Related papers  (2022-02-07T08:46:47Z)
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers [105.74546828182834]
  We show a hardware-efficient dynamic inference regime, named dynamic weight slicing, which adaptively slice a part of network parameters for inputs with diverse difficulty levels. We present dynamic slimmable network (DS-Net) and dynamic slice-able network (DS-Net++) by input-dependently adjusting filter numbers of CNNs and multiple dimensions in both CNNs and transformers.
  arXiv  Detail & Related papers  (2021-09-21T09:57:21Z)
• NullaNet Tiny: Ultra-low-latency DNN Inference Through Fixed-function Combinational Logic [4.119948826527649]
  Field-programmable gate array (FPGA)-based accelerators are gaining traction as a serious contender to replace graphics processing unit/central processing unit-based platforms. This paper presents NullaNet Tiny, a framework for constructing resource and energy-efficient, ultra-low-latency FPGA-based neural network accelerators.
  arXiv  Detail & Related papers  (2021-04-07T00:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.