Hyena Operator for Fast Sequential Recommendation

• URL: http://arxiv.org/abs/2603.25027v1
• Date: Thu, 26 Mar 2026 04:47:58 GMT
• Title: Hyena Operator for Fast Sequential Recommendation
• Abstract summary: Sequential recommendation models, particularly those based on attention, achieve strong accuracy but incur quadratic complexity.<n>Sub-quadratic operators such as Hyena provide efficient alternatives in language modeling.<n>We argue that Hyena faces challenges in recommendation due to limited representation capacity on sparse, long user sequences.<n>We propose HyenaRec, a novel recommender that integrates kernel parameterization with convolutions.
• Score: 35.53585245910816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential recommendation models, particularly those based on attention, achieve strong accuracy but incur quadratic complexity, making long user histories prohibitively expensive. Sub-quadratic operators such as Hyena provide efficient alternatives in language modeling, but their potential in recommendation remains underexplored. We argue that Hyena faces challenges in recommendation due to limited representation capacity on sparse, long user sequences. To address these challenges, we propose HyenaRec, a novel sequential recommender that integrates polynomial-based kernel parameterization with gated convolutions. Specifically, we design convolutional kernels using Legendre orthogonal polynomials, which provides a smooth and compact basis for modeling long-term temporal dependencies. A complementary gating mechanism captures fine-grained short-term behavioral bursts, yielding a hybrid architecture that balances global temporal evolution with localized user interests under sparse feedback. This construction enhances expressiveness while scaling linearly with sequence length. Extensive experiments on multiple real-world datasets demonstrate that HyenaRec consistently outperforms Attention-, Recurrent-, and other baselines in ranking accuracy. Moreover, it trains significantly faster (up to 6x speedup), with particularly pronounced advantages on long-sequence scenarios where efficiency is maintained without sacrificing accuracy. These results highlight polynomial-based kernel parameterization as a principled and scalable alternative to attention for sequential recommendation.

Related papers

• Preference-Drift-Aware Subsequence Learning and Hierarchical Context Fusion for Long-Sequence Generative Recommendation [20.33597628832828]
  Long-sequence generative recommendation methods autoregressively model the user's interaction sequence to generate the next-item representation.<n>Existing methods fall into two categories: efficient full-sequence modeling and target-aware context retrieval.<n>We propose a preference-drift-aware subsequence learning and hierarchical context fusion for long-sequence generative recommendation.
  arXiv  Detail & Related papers  (2026-09-11T08:01:29Z)
• Gradient-Free Noise Optimization for Reward Alignment in Generative Models [52.42087778653147]
  ZeNO is a gradient-free framework that formulates noise optimization as a path-integral control problem.<n>It enables effective inference-time scaling and demonstrates strong performance across diverse generators and reward functions.
  arXiv  Detail & Related papers  (2026-05-12T00:05:36Z)
• FuXi-Linear: Unleashing the Power of Linear Attention in Long-term Time-aware Sequential Recommendation [86.55349738440087]
  FuXi-Linear is a linear-complexity model designed for efficient long-sequence recommendation.<n>Our approach introduces two key components: (1) a Temporal Retention Channel that independently computes periodic attention weights using temporal data, preventing crosstalk between temporal and semantic signals; and (2) a Linear Positional Channel that integrates positional information through learnable kernels within linear complexity.
  arXiv  Detail & Related papers  (2026-02-27T04:38:28Z)
• HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation [5.1321456889159425]
  HyTRec is a model featuring a Hybrid Attention architecture that decouples long-term stable preferences from short-term intent spikes.<n>Our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions.<n> Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-02-20T15:11:40Z)
• GEMs: Breaking the Long-Sequence Barrier in Generative Recommendation with a Multi-Stream Decoder [54.64137490632567]
  We propose a novel and unified framework designed to capture users' sequences from long-term history.<n>Generative Multi-streamers ( GEMs) break user sequences into three streams.<n>Extensive experiments on large-scale industrial datasets demonstrate that GEMs significantly outperforms state-the-art methods in recommendation accuracy.
  arXiv  Detail & Related papers  (2026-02-14T06:42:56Z)
• Gated Rotary-Enhanced Linear Attention for Long-term Sequential Recommendation [14.581838243440922]
  We propose a long-term sequential Recommendation model with Gated Rotary Enhanced Linear Attention (RecGRELA)<n> Specifically, we propose a Rotary-Enhanced Linear Attention (RELA) module to efficiently model long-range dependency.<n>We also introduce a SiLU-based Gated mechanism for RELA to help the model tell if a user behavior shows a short-term, local interest or a real change in their long-term tastes.
  arXiv  Detail & Related papers  (2025-06-16T09:56:10Z)
• FutureFill: Fast Generation from Convolutional Sequence Models [20.70601085553854]
  FutureFill is a general-purpose fast generation method for any sequence prediction algorithm based on convolutional operators.<n>We demonstrate substantial efficiency gains when generating from a deep convolutional sequence prediction model.
  arXiv  Detail & Related papers  (2024-10-02T15:22:08Z)
• Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers [58.5711048151424]
  We introduce SPARSEK Attention, a novel sparse attention mechanism designed to overcome computational and memory obstacles. Our approach integrates a scoring network and a differentiable top-k mask operator, SPARSEK, to select a constant number of KV pairs for each query. Experimental results reveal that SPARSEK Attention outperforms previous sparse attention methods.
  arXiv  Detail & Related papers  (2024-06-24T15:55:59Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Learning Sequence Representations by Non-local Recurrent Neural Memory [61.65105481899744]
  We propose a Non-local Recurrent Neural Memory (NRNM) for supervised sequence representation learning. Our model is able to capture long-range dependencies and latent high-level features can be distilled by our model. Our model compares favorably against other state-of-the-art methods specifically designed for each of these sequence applications.
  arXiv  Detail & Related papers  (2022-07-20T07:26:15Z)
• Flexible Triggering Kernels for Hawkes Process Modeling [11.90725359131405]
  Recently proposed encoder-decoder structures for modeling Hawkes processes use transformer-inspired architectures. We introduce an efficient and general encoding of the historical event sequence by replacing the complex (multilayered) attention structures with triggering kernels.
  arXiv  Detail & Related papers  (2022-02-03T22:02:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.