Graph Signal Processing Meets Mamba2: Adaptive Filter Bank via Delta Modulation

• URL: http://arxiv.org/abs/2603.22333v1
• Date: Fri, 20 Mar 2026 23:39:13 GMT
• Title: Graph Signal Processing Meets Mamba2: Adaptive Filter Bank via Delta Modulation
• Abstract summary: State-space models (SSMs) offer efficient alternatives to attention with linear-time recurrence.<n>Mamba2, a recent SSM-based language model, uses selective input gating and a multi-head structure.<n>We propose a novel method called Hierarchical ADaptive filter bank for Efficient SSMs (HADES)
• Score: 30.994664211855607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State-space models (SSMs) offer efficient alternatives to attention with linear-time recurrence. Mamba2, a recent SSM-based language model, uses selective input gating and a multi-head structure, enabling parallel computation and strong benchmark performance. However, its multi-head recurrence operates independently without structured utilization or analysis. In this work, we propose a novel method called Hierarchical ADaptive filter bank for Efficient SSMs (HADES), a Graph Signal Processing (GSP)-inspired framework that reinterprets Mamba2 as an adaptive filter bank on a line graph. Our hierarchical architecture introduces two filter types: shared filters for global low-pass behavior and expert filters for local high-pass behavior, achieved through structured bias on the parameter Δ. HADES achieves comparable performance to baseline models including Mamba2 across various benchmarks in language modeling, commonsense reasoning, and long-context retrieval, while using only 58.9% of the original parameters. In this regard, HADES bridges GSP and neural sequence modeling, enabling efficient, hierarchical, and interpretable filtering within state-space models.

Related papers

• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models [102.20309135516186]
  Cross-entropy (CE) training provides dense and scalable supervision for language models.<n>We introduce a feature-matching objective for language-model fine-tuning.<n>To optimize this objective efficiently, we propose energy-based fine-tuning.
  arXiv  Detail & Related papers  (2026-03-12T17:57:50Z)
• Filtering Jump Markov Systems with Partially Known Dynamics: A Model-Based Deep Learning Approach [33.421237778335076]
  JMFNet is a novel model-based deep learning framework for real-time state-state estimation in jump Markov systems.<n>A hybrid architecture comprising two Recurrent Neural Networks (RNNs) is proposed.<n>The proposed RNNs are trained jointly using an alternating least squares strategy that enables mutual adaptation without supervision of the latent modes.
  arXiv  Detail & Related papers  (2025-11-11T12:11:24Z)
• Gated Associative Memory: A Parallel O(N) Architecture for Efficient Sequence Modeling [0.0]
  Gated Associative Memory (GAM) network is a novel, fully parallel architecture for sequence modeling.<n>We implement GAM from scratch and conduct a rigorous comparative analysis against a standard Transformer model and a modern linear-time baseline.<n>Our experiments demonstrate that GAM is consistently faster, outperforming both baselines on training speed, and achieves a superior or competitive final validation perplexity across all datasets.
  arXiv  Detail & Related papers  (2025-08-30T20:59:46Z)
• Sequential-Parallel Duality in Prefix Scannable Models [68.39855814099997]
  Recent developments have given rise to various models, such as Gated Linear Attention (GLA) and Mamba.<n>This raises a natural question: can we characterize the full class of neural sequence models that support near-constant-time parallel evaluation and linear-time, constant-space sequential inference?
  arXiv  Detail & Related papers  (2025-06-12T17:32:02Z)
• SIGMA: Selective Gated Mamba for Sequential Recommendation [56.85338055215429]
  Mamba, a recent advancement, has exhibited exceptional performance in time series prediction.<n>We introduce a new framework named Selective Gated Mamba ( SIGMA) for Sequential Recommendation.<n>Our results indicate that SIGMA outperforms current models on five real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T09:12:59Z)
• EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation [0.0]
  Control theory emphasizes the use of state space models (SSMs) for managing long-range dependencies. We introduce EchoMamba4Rec, a model for predicting user preferences and sequential dependencies based on historical behavior. We show that EchoMamba significantly outperforms existing models, providing more accurate and personalized recommendations.
  arXiv  Detail & Related papers  (2024-06-04T09:07:58Z)
• Frequency-aware Graph Signal Processing for Collaborative Filtering [26.317108637430664]
  We propose a frequency-aware graph signal processing method (FaGSP) for collaborative filtering. Firstly, we design a Cascaded Filter Module, consisting of an ideal high-pass filter and an ideal low-pass filter. Then, we devise a Parallel Filter Module, consisting of two low-pass filters that can easily capture the hierarchy of neighborhood.
  arXiv  Detail & Related papers  (2024-02-13T12:53:18Z)
• Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference [47.460898983429374]
  We introduce an ensemble Kalman filter (EnKF) into the non-mean-field (NMF) variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO) We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-12-10T15:22:30Z)
• Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler [103.97487121678276]
  Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs. We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
  arXiv  Detail & Related papers  (2023-07-01T02:28:41Z)
• Online Multi-Object Tracking and Segmentation with GMPHD Filter and Mask-based Affinity Fusion [79.87371506464454]
  We propose a fully online multi-object tracking and segmentation (MOTS) method that uses instance segmentation results as an input. The proposed method is based on the Gaussian mixture probability hypothesis density (GMPHD) filter, a hierarchical data association (HDA), and a mask-based affinity fusion (MAF) model. In the experiments on the two popular MOTS datasets, the key modules show some improvements.
  arXiv  Detail & Related papers  (2020-08-31T21:06:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.