Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation

• URL: http://arxiv.org/abs/2603.19742v1
• Date: Fri, 20 Mar 2026 08:28:24 GMT
• Title: Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation
• Abstract summary: We introduce Dual Path Attribution (DPA), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass.<n>DPA achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines.
• Score: 5.186807923082922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding the internal mechanisms of transformer-based large language models (LLMs) is crucial for their reliable deployment and effective operation. While recent efforts have yielded a plethora of attribution methods attempting to balance faithfulness and computational efficiency, dense component attribution remains prohibitively expensive. In this work, we introduce Dual Path Attribution (DPA), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass without requiring counterfactual examples. DPA analytically decomposes and linearizes the computational structure of the SwiGLU Transformers into distinct pathways along which it propagates a targeted unembedding vector to receive the effective representation at each residual position. This target-centric propagation achieves O(1) time complexity with respect to the number of model components, scaling to long input sequences and dense component attribution. Extensive experiments on standard interpretability benchmarks demonstrate that DPA achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines.

Related papers

• Efficient Document Tampering Localization with Multi-Level Discrepancy Features and Unified DCT-Quantization Embedding [9.386011004825281]
  Localizing document tampering is extremely challenging, as manipulations are crafted to appear visually consistent.<n>We propose textbfDiffNet, a relatively simple yet effective RGB--DCT early-fusion architecture driven by two key design choices.<n>Our approach achieves state-of-the-art performance on cross-domain and human-made document tampering localization, outperforming prior methods by around 30%.
  arXiv  Detail & Related papers  (2026-06-21T00:29:59Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Move What Matters: Parameter-Efficient Domain Adaptation via Optimal Transport Flow for Collaborative Perception [8.774658029766988]
  FlowAdapt is a parameter-efficient framework grounded in optimal transport theory.<n>We introduce a Wasserstein Greedy Sampling strategy to selectively filter redundant samples.<n> Progressive Knowledge Transfer module is designed to inject compressed early-stage representations into later stages.
  arXiv  Detail & Related papers  (2026-02-12T04:36:50Z)
• Hybrid Dual-Path Linear Transformations for Efficient Transformer Architectures [0.0]
  We introduce the Hybrid Dual-Path Linear (HDPL) operator, which decomposes the affine transformation into two topologically distinct pathways.<n> Experiments on the FineWeb-Edu dataset demonstrate that the HDPL architecture outperforms a standard Llama-style baseline.<n>We discuss how the explicit materialization of a probabilistic latent space within the Transformer backbone serves as a vital architectural affordance.
  arXiv  Detail & Related papers  (2026-02-05T20:16:10Z)
• LAPA: Log-Domain Prediction-Driven Dynamic Sparsity Accelerator for Transformer Model [14.53308613746613]
  This paper proposes a log-domain attention prediction algorithm-architecture co-design, named LAPA.<n>Results show that LAPA achieves 3.52x, 3.24x and 2.79x higher energy efficiency than the state-of-the-art (SOTA) works Spatten, Sanger and FACT.
  arXiv  Detail & Related papers  (2025-11-26T07:24:48Z)
• DualGazeNet: A Biologically Inspired Dual-Gaze Query Network for Salient Object Detection [52.32976488996896]
  We introduce DualGazeNet, a pure Transformer framework for salient object detection.<n>Experiments on five RGB benchmarks show that DualGazeNet consistently surpasses 25 state-of-the-art CNN- and Transformer-based methods.
  arXiv  Detail & Related papers  (2025-11-24T08:08:22Z)
• Contrast & Compress: Learning Lightweight Embeddings for Short Trajectories [11.6132604160666]
  We propose a novel framework for learning fixed-dimensional embeddings for short trajectories by leveraging a Transformer encoder.<n>We analyze the influence of Cosine and FFT-based similarity metrics within the contrastive learning paradigm.<n>Our empirical evaluation on the Argoverse 2 dataset demonstrates that embeddings shaped by Cosine similarity objectives yield superior clustering of trajectories.
  arXiv  Detail & Related papers  (2025-06-03T07:53:04Z)
• Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability [53.21677928601684]
  Layer-wise relevance propagation is one of the most promising approaches to explainability in deep learning.<n>We propose specialized theoretically-grounded LRP rules designed to propagate attributions across various positional encoding methods.<n>Our method significantly outperforms the state-of-the-art in both vision and NLP explainability tasks.
  arXiv  Detail & Related papers  (2025-06-02T18:07:55Z)
• Fourier Test-time Adaptation with Multi-level Consistency for Robust Classification [10.291631977766672]
  We propose a novel approach called Fourier Test-time Adaptation (FTTA) to integrate input and model tuning. FTTA builds a reliable multi-level consistency measurement of paired inputs for achieving self-supervised of prediction. It was extensively validated on three large classification datasets with different modalities and organs.
  arXiv  Detail & Related papers  (2023-06-05T02:29:38Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection [77.50110439560152]
  Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF) We propose a novel and efficient context modeling mechanism that can help existing FPs deliver better MFF results. In particular, we introduce a novel insight that comprehensive contexts can be decomposed and condensed into two types of representations for higher efficiency.
  arXiv  Detail & Related papers  (2022-07-14T01:45:03Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.