Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey

• URL: http://arxiv.org/abs/2609.01212v1
• Date: Tue, 01 Sep 2026 13:18:00 GMT
• Title: Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey
• Abstract summary: We investigate the most recent advances, trends, and design choices for Transformer inference on FPGA platforms.<n>We perform a systematic literature review, extracting and delving into preferred techniques for implementation and optimisation.<n>This study and the provided taxonomy of topics could act as a guide for researchers from the academia and industry alike.
• Score: 0.2851702684899107
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rapid and continuous growth in the incorporation of machine learning models based on the Transformer architecture, capable deployment is in high demand. In this context, capable deployment refers to operational performance aspects, e.g., throughput and latency, as well as efficiency aspects, e.g., energy consumption. When it comes to the task of inference using such models, purpose-built hardware accelerators provide a lucrative alternative to common deployment choices, such as Central Processing Units (CPUs) and Graphics Processing Units (GPUs). The Field Programmable Gate Array (FPGA) platforms category is an example of such alternative accelerators, promising implementation flexibility, energy efficiency, improved latency and suitability for on-site deployment. We investigate the most recent advances, trends, and design choices for Transformer inference on FPGA platforms. We perform a systematic literature review, extracting and delving into preferred techniques for implementation and optimisation. This study and the provided taxonomy of topics could act as a guide for researchers from the academia and industry alike.

Related papers

• Architectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review [0.7874708385247353]
  Deep learning (DL) has emerged as a rapidly developing advanced technology.<n>This article explores various hardware level optimizations for DL.<n>It provides an overview of state-of-the-art FPGA-based neural network accelerators.
  arXiv  Detail & Related papers  (2026-02-25T17:53:35Z)
• FPGA or GPU? Analyzing comparative research for application-specific guidance [0.0]
  This paper synthesizes insights from various research articles to guide users in selecting the appropriate accelerator for domain-specific applications.<n>By categorizing the reviewed studies and analyzing key performance metrics, this work highlights the strengths, limitations, and ideal use cases for FPGAs and GPU.<n>The findings offer actionable recommendations, helping researchers and practitioners navigate trade-offs in performance, energy efficiency, and programmability.
  arXiv  Detail & Related papers  (2025-11-09T22:54:28Z)
• TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs [0.2851702684899107]
  We aim to develop tools for monolithic, or partitioned Transformer synthesis, specifically targeting inference.<n>Our primary use-case involves two machine learning model designs for tracking, derived from the TrackFormers project.
  arXiv  Detail & Related papers  (2025-09-30T14:44:43Z)
• FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review [3.7810245817090906]
  Convolutional Neural Networks (CNNs) are fundamental to deep learning, driving applications across various domains.<n>This paper provides a comprehensive review of FPGA-based hardware accelerators specifically designed for CNNs.
  arXiv  Detail & Related papers  (2025-05-04T04:03:37Z)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models [50.40325411764262]
  Large-scale Mixture of Experts (MoE) models offer enhanced model capacity and computational efficiency through conditional computation.<n> deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency.<n>This survey analyzes optimization techniques for MoE models across the entire system stack.
  arXiv  Detail & Related papers  (2024-12-18T14:11:15Z)
• AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer [54.713778961605115]
  Vision Transformer (ViT) has become one of the most prevailing fundamental backbone networks in the computer vision community. We propose a novel non-uniform quantizer, dubbed the Adaptive Logarithm AdaLog (AdaLog) quantizer.
  arXiv  Detail & Related papers  (2024-07-17T18:38:48Z)
• Inference Optimization of Foundation Models on AI Accelerators [68.24450520773688]
  Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI. As the number of model parameters reaches to hundreds of billions, their deployment incurs prohibitive inference costs and high latency in real-world scenarios. This tutorial offers a comprehensive discussion on complementary inference optimization techniques using AI accelerators.
  arXiv  Detail & Related papers  (2024-07-12T09:24:34Z)
• Enhancing Dropout-based Bayesian Neural Networks with Multi-Exit on FPGA [20.629635991749808]
  This paper proposes an algorithm and hardware co-design framework that can generate field-programmable gate array (FPGA)-based accelerators for efficient BayesNNs. At the algorithm level, we propose novel multi-exit dropout-based BayesNNs with reduced computational and memory overheads. At the hardware level, this paper introduces a transformation framework that can generate FPGA-based accelerators for the proposed efficient BayesNNs.
  arXiv  Detail & Related papers  (2024-06-20T17:08:42Z)
• Benchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processors [5.432613942292548]
  Transformer models have achieved remarkable success in various machine learning tasks but suffer from high computational complexity and resource requirements. Specialized AI hardware accelerators, such as the Habana GAUDI architecture, offer a promising solution to tackle these issues. This paper explores the untapped potential of using GAUDI processors to accelerate Transformer-based models, addressing key challenges in the process.
  arXiv  Detail & Related papers  (2023-09-29T04:49:35Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• Full Stack Optimization of Transformer Inference: a Survey [58.55475772110702]
  Transformer models achieve superior accuracy across a wide range of applications. The amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate. There has been an increased focus on making Transformer models more efficient.
  arXiv  Detail & Related papers  (2023-02-27T18:18:13Z)
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration [130.89746032163106]
  We propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration. We present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
  arXiv  Detail & Related papers  (2020-11-10T19:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.