Vectorized FlashAttention with Low-cost Exponential Computation in RISC-V Vector Processors

• URL: http://arxiv.org/abs/2510.06834v1
• Date: Wed, 08 Oct 2025 09:55:32 GMT
• Title: Vectorized FlashAttention with Low-cost Exponential Computation in RISC-V Vector Processors
• Authors: Vasileios Titopoulos, Kosmas Alexandridis, Giorgos Dimitrakopoulos,
• Abstract summary: This work focuses on the acceleration of attention kernel using FlashAttention algorithm, in vector processors.<n>By utilizing a low-cost approximation for exponentials in floating-point arithmetic, we reduce the cost of computing the exponential function.
• Score: 5.385189465543017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Attention is a core operation in numerous machine learning and artificial intelligence models. This work focuses on the acceleration of attention kernel using FlashAttention algorithm, in vector processors, particularly those based on the RISC-V instruction set architecture (ISA). This work represents the first effort to vectorize FlashAttention, minimizing scalar code and simplifying the computational complexity of evaluating exponentials needed by softmax used in attention. By utilizing a low-cost approximation for exponentials in floating-point arithmetic, we reduce the cost of computing the exponential function without the need to extend baseline vector ISA with new custom instructions. Also, appropriate tiling strategies are explored with the goal to improve memory locality. Experimental results highlight the scalability of our approach, demonstrating significant performance gains with the vectorized implementations when processing attention layers in practical applications.

Related papers

• Customizing the Inductive Biases of Softmax Attention using Structured Matrices [46.30740502186753]
  Core component of attention is the scoring function, which transforms the inputs into low-dimensional queries and keys.<n>We propose new scoring functions based on computationally efficient structured matrices with high ranks, including Block-Train (BTT) and Multi-Level Low Rank (MLR)<n>Our MLR-based attention method achieves improved scaling laws compared to both standard attention and variants of sliding window attention.
  arXiv  Detail & Related papers  (2025-09-09T17:50:58Z)
• Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators [3.668018928502405]
  We focus on optimizing the kernel of floating-point-based FlashAttention using new hardware operators that fuse the computation of exponentials and vector multiplications.<n>The proposed ExpMul hardware operators significantly reduce the area and power costs of FlashAttention-based hardware accelerators.
  arXiv  Detail & Related papers  (2025-05-20T13:00:59Z)
• FLASH-D: FlashAttention with Hidden Softmax Division [3.668018928502405]
  Building on online softmax computation, FlashAttention integrates softmax calculation with matrix arithmetic.<n>This work presents FLASH-D a mathematically equivalent, yet simplified, formulation that achieves: (a) hiding softmax division within other non-linear function evaluations; (b) inherently numerically stable computation of exponentials; and (c) a reduction in computational cost without introducing numerical approximations to the FlashAttention kernel.
  arXiv  Detail & Related papers  (2025-05-20T11:01:33Z)
• FlashBias: Fast Computation of Attention with Bias [70.44379606190569]
  Attention with bias has been widely deployed in vision, language, protein-folding and other advanced scientific models.<n>It disrupts the tightly fused memory-compute pipeline that underlies the speed of accelerators like FlashAttention.<n>This paper presents FlashBias based on the low-rank compressed sensing theory, which can provide fast-exact computation for many widely used attention biases.
  arXiv  Detail & Related papers  (2025-05-17T15:12:50Z)
• MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices [24.1144641404561]
  We propose a scheme for exact attention inference acceleration on memory-constrained edge accelerators.<n>We show up to 2.75x speedup and 54% reduction in energy consumption as compared to the state-of-the-art attention fusion method (FLAT) in the edge computing scenario.
  arXiv  Detail & Related papers  (2024-11-20T19:44:26Z)
• Dynamic Range Reduction via Branch-and-Bound [1.0141085397402314]
  Key strategy to enhance hardware accelerators is the reduction of precision in arithmetic operations.<n>This paper introduces a fully principled Branch-and-Bound algorithm for reducing precision needs in QUBO problems.<n> Experiments validate our algorithm's effectiveness on an actual quantum annealer.
  arXiv  Detail & Related papers  (2024-09-17T03:07:56Z)
• Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers [58.5711048151424]
  We introduce SPARSEK Attention, a novel sparse attention mechanism designed to overcome computational and memory obstacles. Our approach integrates a scoring network and a differentiable top-k mask operator, SPARSEK, to select a constant number of KV pairs for each query. Experimental results reveal that SPARSEK Attention outperforms previous sparse attention methods.
  arXiv  Detail & Related papers  (2024-06-24T15:55:59Z)
• Efficient Model-Free Exploration in Low-Rank MDPs [76.87340323826945]
  Low-Rank Markov Decision Processes offer a simple, yet expressive framework for RL with function approximation. Existing algorithms are either (1) computationally intractable, or (2) reliant upon restrictive statistical assumptions. We propose the first provably sample-efficient algorithm for exploration in Low-Rank MDPs.
  arXiv  Detail & Related papers  (2023-07-08T15:41:48Z)
• Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RL [106.82295532402335]
  Existing reinforcement learning algorithms suffer from computational intractability, strong statistical assumptions, and suboptimal sample complexity. We provide the first computationally efficient algorithm that attains rate-optimal sample complexity with respect to the desired accuracy level. Our algorithm, MusIK, combines systematic exploration with representation learning based on multi-step inverse kinematics.
  arXiv  Detail & Related papers  (2023-04-12T14:51:47Z)
• Skip-Attention: Improving Vision Transformers by Paying Less Attention [55.47058516775423]
  Vision computation transformers (ViTs) use expensive self-attention operations in every layer. We propose SkipAt, a method to reuse self-attention from preceding layers to approximate attention at one or more subsequent layers. We show the effectiveness of our method in image classification and self-supervised learning on ImageNet-1K, semantic segmentation on ADE20K, image denoising on SIDD, and video denoising on DAVIS.
  arXiv  Detail & Related papers  (2023-01-05T18:59:52Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Efficient Non-Local Contrastive Attention for Image Super-Resolution [48.093500219958834]
  Non-Local Attention (NLA) brings significant improvement for Single Image Super-Resolution (SISR) by leveraging intrinsic feature correlation in natural images. We propose a novel Efficient Non-Local Contrastive Attention (ENLCA) to perform long-range visual modeling and leverage more relevant non-local features.
  arXiv  Detail & Related papers  (2022-01-11T05:59:09Z)
• Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification [70.50751397870972]
  FastGAT is a method to make attention based GNNs lightweight by using spectral sparsification to generate an optimal pruning of the input graph. We experimentally evaluate FastGAT on several large real world graph datasets for node classification tasks.
  arXiv  Detail & Related papers  (2020-06-15T22:07:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.