Mamba Knockout for Unraveling Factual Information Flow

• URL: http://arxiv.org/abs/2505.24244v1
• Date: Fri, 30 May 2025 06:08:36 GMT
• Title: Mamba Knockout for Unraveling Factual Information Flow
• Authors: Nir Endy, Idan Daniel Grosbard, Yuval Ran-Milo, Yonatan Slutzky, Itay Tshuva, Raja Giryes,
• Abstract summary: We rely on theoretical and empirical connections to Transformer-based architectures and their attention mechanisms.<n>We adapt attentional interpretability techniques originally developed for Transformers to both Mamba-1 and Mamba-2.<n>By further leveraging Mamba's structured factorization, we disentangle how distinct "features" either enable token-to-token information exchange or enrich individual tokens.
• Score: 22.520634805939093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the flow of factual information in Mamba State-Space Model (SSM)-based language models. We rely on theoretical and empirical connections to Transformer-based architectures and their attention mechanisms. Exploiting this relationship, we adapt attentional interpretability techniques originally developed for Transformers--specifically, the Attention Knockout methodology--to both Mamba-1 and Mamba-2. Using them we trace how information is transmitted and localized across tokens and layers, revealing patterns of subject-token information emergence and layer-wise dynamics. Notably, some phenomena vary between mamba models and Transformer based models, while others appear universally across all models inspected--hinting that these may be inherent to LLMs in general. By further leveraging Mamba's structured factorization, we disentangle how distinct "features" either enable token-to-token information exchange or enrich individual tokens, thus offering a unified lens to understand Mamba internal operations.

Related papers

• Differential Mamba [16.613266337054267]
  Sequence models like Transformers and RNNs often overallocate attention to irrelevant context, leading to noisy intermediate representations.<n>Recent work has shown that differential design can mitigate this issue in Transformers, improving their effectiveness across various applications.<n>We show that a naive adaptation of differential design to Mamba is insufficient and requires careful architectural modifications.
  arXiv  Detail & Related papers  (2025-07-08T17:30:14Z)
• LaTIM: Measuring Latent Token-to-Token Interactions in Mamba Models [1.249658136570244]
  State space models (SSMs) have emerged as an efficient alternative to transformers for long-context sequence modeling.<n>SSMs lack the interpretability tools that have been crucial for understanding and improving attention-based architectures.<n>We introduce LaTIM, a novel token-level decomposition method for both Mamba-1 and Mamba-2 that enables fine-grained interpretability.
  arXiv  Detail & Related papers  (2025-02-21T17:33:59Z)
• TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba [88.31117598044725]
  We explore cross-architecture training to transfer the ready knowledge in existing Transformer models to alternative architecture Mamba, termed TransMamba.<n>Our approach employs a two-stage strategy to expedite training new Mamba models, ensuring effectiveness in across uni-modal and cross-modal tasks.<n>For cross-modal learning, we propose a cross-Mamba module that integrates language awareness into Mamba's visual features, enhancing the cross-modal interaction capabilities of Mamba architecture.
  arXiv  Detail & Related papers  (2025-02-21T01:22:01Z)
• From Markov to Laplace: How Mamba In-Context Learns Markov Chains [36.22373318908893]
  We study in-context learning on Markov chains and uncover a surprising phenomenon.<n>Unlike transformers, even a single-layer Mamba efficiently learns the in-context Laplacian smoothing estimator.<n>These theoretical insights align strongly with empirical results and represent the first formal connection between Mamba and optimal statistical estimators.
  arXiv  Detail & Related papers  (2025-02-14T14:13:55Z)
• Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures [49.24097977047392]
  We investigate two mainstream architectures for language modeling, namely Transformers and Mambas, to explore the extent of their mechanistic similarity. We propose to use Sparse Autoencoders (SAEs) to isolate interpretable features from these models and show that most features are similar in these two models.
  arXiv  Detail & Related papers  (2024-10-09T08:28:53Z)
• Demystify Mamba in Vision: A Linear Attention Perspective [72.93213667713493]
  Mamba is an effective state space model with linear computation complexity.<n>We show that Mamba shares surprising similarities with linear attention Transformer.<n>We propose a Mamba-Inspired Linear Attention (MILA) model by incorporating the merits of these two key designs into linear attention.
  arXiv  Detail & Related papers  (2024-05-26T15:31:09Z)
• MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities [72.05167902805405]
  We present MergeNet, which learns to bridge the gap of parameter spaces of heterogeneous models.<n>The core mechanism of MergeNet lies in the parameter adapter, which operates by querying the source model's low-rank parameters.<n> MergeNet is learned alongside both models, allowing our framework to dynamically transfer and adapt knowledge relevant to the current stage.
  arXiv  Detail & Related papers  (2024-04-20T08:34:39Z)
• Locating and Editing Factual Associations in Mamba [22.097117651225595]
  We investigate the mechanisms of factual recall in the Mamba state space model. We compare Mamba directly to a similar-sized autoregressive transformer LM.
  arXiv  Detail & Related papers  (2024-04-04T17:58:31Z)
• Is Mamba Capable of In-Context Learning? [63.682741783013306]
  State of the art foundation models such as GPT-4 perform surprisingly well at in-context learning (ICL) This work provides empirical evidence that Mamba, a newly proposed state space model, has similar ICL capabilities.
  arXiv  Detail & Related papers  (2024-02-05T16:39:12Z)
• Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective [106.92016199403042]
  We empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. We employ sensitivity-based techniques to extract and align knowledge-specific parameters between different large language models. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer.
  arXiv  Detail & Related papers  (2023-10-17T17:58:34Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.