TRINE: A Token-Aware, Runtime-Adaptive FPGA Inference Engine for Multimodal AI

• URL: http://arxiv.org/abs/2603.22867v1
• Date: Tue, 24 Mar 2026 07:12:15 GMT
• Title: TRINE: A Token-Aware, Runtime-Adaptive FPGA Inference Engine for Multimodal AI
• Abstract summary: TRINE executes end-to-end multimodal inference without reconfiguration.<n> token pruning alone yields up to 7.8x on ViT-heavy pipelines.<n>With int8 quantization, accuracy drops remain 2.5% across representative tasks.
• Score: 16.724696664451653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal stacks that mix ViTs, CNNs, GNNs, and transformer NLP strain embedded platforms because their compute/memory patterns diverge and hard real-time targets leave little slack. TRINE is a single-bitstream FPGA accelerator and compiler that executes end-to-end multimodal inference without reconfiguration. Layers are unified as DDMM/SDDMM/SpMM and mapped to a mode-switchable engine that toggles at runtime among weight/output-stationary systolic, 1xCS SIMD, and a routable adder tree (RADT) on a shared PE array. A width-matched, two-stage top-k unit enables in-stream token pruning, while dependency-aware layer offloading (DALO) overlaps independent kernels across reconfigurable processing units to sustain utilization. Evaluated on Alveo U50 and ZCU104, TRINE reduces latency by up to 22.57x vs. RTX 4090 and 6.86x vs. Jetson Orin Nano at 20-21 W; token pruning alone yields up to 7.8x on ViT-heavy pipelines, and DALO contributes up to 79% throughput improvement. With int8 quantization, accuracy drops remain <2.5% across representative tasks, delivering state-of-the-art latency and energy efficiency for unified vision, language, and graph workloads-in one bitstream.

Related papers

• SymbolicLight V2: Hybrid Neuromorphic Architecture and Sparse Execution for Low-Energy Language Inference [8.419155861590548]
  SymbolicLight V2 combines sparse event computation with continuous-state processing.<n>We implement SymbolicLight V2 on an Alveo U50C FPGA using digital fixed-point arithmetic.<n>On four Cortex-A76 cores of a ROCK 5T, complete requests reach 65.4 tokens/s at 9.80 W and 0.151 J per generated token at the adapter's AC input.
  arXiv  Detail & Related papers  (2026-09-09T06:18:49Z)
• TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation [105.14745366152725]
  TurboT2VA is a distillation and inference framework for joint text-to-video-audio generation.<n>On LTX-2, four-step distillation reduces generator latency from 50.52s to 2.51s at the standard evaluation resolution of 512$times$768.<n>The complete stack reduces generator latency from 318.74s to 5.83s on one NVIDIA H20, achieving a 54.67$times$ generator-only speedup.
  arXiv  Detail & Related papers  (2026-08-25T15:09:02Z)
• CARMEN: CORDIC-Accelerated Resource-Efficient Multi-Precision Inference Engine for Deep Learning [0.13048920509133807]
  This paper presents a runtime-adaptive, CORDIC-accelerated multi-precision vector engine for resource-efficient deep learning inference.<n>The architecture integrates a low-resource iterative CORDIC-based MAC unit with a time-multiplexed multi-activation function block.<n>FPGA deployment on PynqZ2 validates 154.6 ms latency at 0.43 W for real-time object detection.
  arXiv  Detail & Related papers  (2026-05-07T19:25:21Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• QUILL: An Algorithm-Architecture Co-Design for Cache-Local Deformable Attention [12.542462936966844]
  QUILL is a schedule-aware accelerator that turns deformable attention into cache-friendly, single-pass work.<n>A fused MSDeformAttn engine executes, Softmax, aggregation, and the final projection (W''m) in one pass without spilling intermediates.
  arXiv  Detail & Related papers  (2025-11-17T18:34:04Z)
• Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing [67.98609858326951]
  Intra-DP is a high-performance collaborative inference system optimized for deep neural networks (DNNs) on mobile devices.<n>It reduces per-inference latency by up to 50% and energy consumption by up to 75% compared to state-of-the-art baselines.<n>The evaluation demonstrates that Intra-DP reduces per-inference latency by up to 50% and energy consumption by up to 75% compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-07-08T09:50:57Z)
• A Runtime-Adaptive Transformer Neural Network Accelerator on FPGAs [0.0]
  ADAPTOR is a runtime-adaptive accelerator for dense matrix computations in transformer encoders and decoders on FPGAs.<n>It incorporates efficient matrix tiling to distribute resources across FPGA platforms.<n>It achieves a speedup of 1.7 to 2.25$times$ compared to some state-of-the-art FPGA-based accelerators.
  arXiv  Detail & Related papers  (2024-11-27T08:53:19Z)
• ProTEA: Programmable Transformer Encoder Acceleration on FPGA [0.0]
  Transformer neural networks (TNNs) have been widely utilized on a diverse range of applications, including natural language processing (NLP), machine translation, and computer vision (CV) Despite the popularity of TNNs, there has been limited numbers of hardware accelerators targeting these two critical blocks. This paper introduces textitProTEA, a programmable runtime accelerator tailored for the dense computations of state-of-the-art transformer encoders.
  arXiv  Detail & Related papers  (2024-09-21T01:44:13Z)
• Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers [56.37495946212932]
  Vision transformers (ViTs) have demonstrated their superior accuracy for computer vision tasks compared to convolutional neural networks (CNNs) This work proposes Quasar-ViT, a hardware-oriented quantization-aware architecture search framework for ViTs.
  arXiv  Detail & Related papers  (2024-07-25T16:35:46Z)
• Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts [60.1586169973792]
  M$3$ViT is the latest multi-task ViT model that introduces mixture-of-experts (MoE) MoE achieves better accuracy and over 80% reduction computation but leaves challenges for efficient deployment on FPGA. Our work, dubbed Edge-MoE, solves the challenges to introduce the first end-to-end FPGA accelerator for multi-task ViT with a collection of architectural innovations.
  arXiv  Detail & Related papers  (2023-05-30T02:24:03Z)
• LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics [45.666822327616046]
  This work presents a novel reconfigurable architecture for Low Graph Neural Network (LL-GNN) designs for particle detectors. The LL-GNN design advances the next generation of trigger systems by enabling sophisticated algorithms to process experimental data efficiently.
  arXiv  Detail & Related papers  (2022-09-28T12:55:35Z)
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference [82.1584439276834]
  Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. We present EdgeBERT, an in-depth algorithm- hardware co-design for latency-aware energy optimization for multi-task NLP.
  arXiv  Detail & Related papers  (2020-11-28T19:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.