FMM-X3D: FPGA-based modeling and mapping of X3D for Human Action Recognition

• URL: http://arxiv.org/abs/2305.18479v1
• Date: Mon, 29 May 2023 11:17:51 GMT
• Title: FMM-X3D: FPGA-based modeling and mapping of X3D for Human Action Recognition
• Authors: Petros Toupas, Christos-Savvas Bouganis, Dimitrios Tzovaras
• Abstract summary: This paper addresses the problem of mapping X3D, a state-of-the-art model in Human Action Recognition, onto any FPGA device. The proposed toolflow generates an optimised stream-based hardware system, taking into account the available resources and off-chip memory characteristics of the FPGA device.
• Score: 10.385864925381384
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D Convolutional Neural Networks are gaining increasing attention from researchers and practitioners and have found applications in many domains, such as surveillance systems, autonomous vehicles, human monitoring systems, and video retrieval. However, their widespread adoption is hindered by their high computational and memory requirements, especially when resource-constrained systems are targeted. This paper addresses the problem of mapping X3D, a state-of-the-art model in Human Action Recognition that achieves accuracy of 95.5\% in the UCF101 benchmark, onto any FPGA device. The proposed toolflow generates an optimised stream-based hardware system, taking into account the available resources and off-chip memory characteristics of the FPGA device. The generated designs push further the current performance-accuracy pareto front, and enable for the first time the targeting of such complex model architectures for the Human Action Recognition task.

Related papers

• Enhancing Dropout-based Bayesian Neural Networks with Multi-Exit on FPGA [20.629635991749808]
  This paper proposes an algorithm and hardware co-design framework that can generate field-programmable gate array (FPGA)-based accelerators for efficient BayesNNs. At the algorithm level, we propose novel multi-exit dropout-based BayesNNs with reduced computational and memory overheads. At the hardware level, this paper introduces a transformation framework that can generate FPGA-based accelerators for the proposed efficient BayesNNs.
  arXiv  Detail & Related papers  (2024-06-20T17:08:42Z)
• SATAY: A Streaming Architecture Toolflow for Accelerating YOLO Models on FPGA Devices [48.47320494918925]
  This work tackles the challenges of deploying stateof-the-art object detection models onto FPGA devices for ultralow latency applications. We employ a streaming architecture design for our YOLO accelerators, implementing the complete model on-chip in a deeply pipelined fashion. We introduce novel hardware components to support the operations of YOLO models in a dataflow manner, and off-chip memory buffering to address the limited on-chip memory resources.
  arXiv  Detail & Related papers  (2023-09-04T13:15:01Z)
• fpgaHART: A toolflow for throughput-oriented acceleration of 3D CNNs for HAR onto FPGAs [10.385864925381384]
  This study proposes a toolflow that optimises the mapping of 3D CNN models for Human Action Recognition onto FPGA devices. The proposed system employs Synchronous Dataflow (SDF) graphs to model the designs and introduces transformations to expand and explore the design space. A variety of 3D CNN models were evaluated using the proposed toolflow on multiple FPGA devices, demonstrating its potential to deliver competitive performance.
  arXiv  Detail & Related papers  (2023-05-31T14:30:17Z)
• HARFLOW3D: A Latency-Oriented 3D-CNN Accelerator Toolflow for HAR on FPGA Devices [71.45672882756001]
  This study introduces a novel streaming architecture based toolflow for mapping 3D Convolutional Neural Networks onto FPGAs. The HARFLOW3D toolflow takes as input a 3D CNN in ONNX format and a description of the FPGA characteristics. The ability of the toolflow to support a broad range of models and devices is shown through a number of experiments on various 3D CNN and FPGA system pairs.
  arXiv  Detail & Related papers  (2023-03-30T08:25:27Z)
• FPGA-optimized Hardware acceleration for Spiking Neural Networks [69.49429223251178]
  This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources. It reduces the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.
  arXiv  Detail & Related papers  (2022-01-18T13:59:22Z)
• Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device [53.323878851563414]
  We propose a compiler-aware unified framework incorporating network enhancement and pruning search with the reinforcement learning techniques. Specifically, a generator Recurrent Neural Network (RNN) is employed to provide the unified scheme for both network enhancement and pruning search automatically. The proposed framework achieves real-time 3D object detection on mobile devices with competitive detection performance.
  arXiv  Detail & Related papers  (2020-12-26T19:41:15Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)
• ASFD: Automatic and Scalable Face Detector [129.82350993748258]
  We propose a novel Automatic and Scalable Face Detector (ASFD) ASFD is based on a combination of neural architecture search techniques as well as a new loss design. Our ASFD-D6 outperforms the prior strong competitors, and our lightweight ASFD-D0 runs at more than 120 FPS with Mobilenet for VGA-resolution images.
  arXiv  Detail & Related papers  (2020-03-25T06:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.