Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs

• URL: http://arxiv.org/abs/2402.07321v1
• Date: Sun, 11 Feb 2024 22:58:49 GMT
• Title: Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
• Abstract summary: We focus on the most basic form of this task -- factual recall. We find that the mechanistic story behind factual recall is more complex than previously thought.
• Score: 1.5571776694273143
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How do transformer-based large language models (LLMs) store and retrieve knowledge? We focus on the most basic form of this task -- factual recall, where the model is tasked with explicitly surfacing stored facts in prompts of form `Fact: The Colosseum is in the country of'. We find that the mechanistic story behind factual recall is more complex than previously thought. It comprises several distinct, independent, and qualitatively different mechanisms that additively combine, constructively interfering on the correct attribute. We term this generic phenomena the additive motif: models compute through summing up multiple independent contributions. Each mechanism's contribution may be insufficient alone, but summing results in constructive interfere on the correct answer. In addition, we extend the method of direct logit attribution to attribute an attention head's output to individual source tokens. We use this technique to unpack what we call `mixed heads' -- which are themselves a pair of two separate additive updates from different source tokens.

Related papers

• Geometric Factual Recall in Transformers [57.48371649045765]
  A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring memorization counts that scale linearly with the number of facts.<n>We develop a theoretical and empirical account of an alternative, emphgeometric form of memorization in which learned embeddings encode structure directly.<n>In a controlled setting where a single-layer transformer must memorize random vectors from subjects to a shared attribute set, we prove that a logarithmic embedding dimension suffices.<n>We extend these results to the multi-hop setting, providing constructions with and without chain-of-thought that exhibit a provable
  arXiv  Detail & Related papers  (2026-05-12T17:22:22Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs [69.28193153685893]
  Large Language Models (LLMs) frequently exhibit strong translation abilities, even without task-specific fine-tuning.<n>To demystify this process, we leverage Sparse Autoencoders (SAEs) and introduce a novel framework for identifying task-specific features.<n>Our work not only decodes a core component of the translation mechanism in LLMs but also provides a blueprint for using internal model mechanism to create more robust and efficient models.
  arXiv  Detail & Related papers  (2026-01-16T06:29:07Z)
• Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context [33.223631694438]
  A key component of in-context reasoning is the ability of language models (LMs) to bind entities for later retrieval.<n>We show that this mechanism generalizes poorly to more complex settings.<n>We develop a causal model combining all three mechanisms that estimates next token distributions with 95% agreement.
  arXiv  Detail & Related papers  (2025-10-07T17:44:30Z)
• Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queries [21.28307426740275]
  To answer one-to-many factual queries, a language model (LM) must simultaneously recall knowledge and avoid repeating previous answers. Across multiple datasets and models, we identify a promote-then-suppress mechanism: the model first recalls all answers, and then suppresses previously generated ones.
  arXiv  Detail & Related papers  (2025-02-27T19:23:15Z)
• Core Context Aware Transformers for Long Context Language Modeling [50.774702091154204]
  We propose a plug-and-play Core Context Aware (CCA) Attention for efficient long-context modeling.<n>Our method automatically focuses and strengthens core context while diminishing redundancy during the learning process.<n>Our method is able to replace the self-attention module in existing Large Language Models with minimal fine-tuning cost.
  arXiv  Detail & Related papers  (2024-12-17T01:54:08Z)
• Counterfactual Generation from Language Models [64.55296662926919]
  We show that counterfactual reasoning is conceptually distinct from interventions. We propose a framework for generating true string counterfactuals. Our experiments demonstrate that the approach produces meaningful counterfactuals.
  arXiv  Detail & Related papers  (2024-11-11T17:57:30Z)
• IIU: Independent Inference Units for Knowledge-based Visual Question Answering [7.3787088958663665]
  We propose Independent Inference Units (IIU) for fine-grained multi-modal reasoning. IIU decomposes intra-modal information by the functionally independent units. Our model achieves a new state-of-the-art, enhancing performance by 3%, and surpassing basic pretrained multi-modal models.
  arXiv  Detail & Related papers  (2024-08-15T07:30:47Z)
• Beyond Single-Feature Importance with ICECREAM [0.4970364068620607]
  We propose an information-theoretic measure for the influence of a coalition of variables on the distribution of a target variable. In experiments with synthetic and real-world data, we show that ICECREAM outperforms state-of-the-art methods for explainability and root cause analysis.
  arXiv  Detail & Related papers  (2023-07-19T06:48:33Z)
• Multimodal Fusion Interactions: A Study of Human and Automatic Quantification [116.55145773123132]
  We study how humans annotate two categorizations of multimodal interactions. We propose a method to automatically convert annotations of partial and counterfactual labels to information decomposition.
  arXiv  Detail & Related papers  (2023-06-07T03:44:50Z)
• EquiMod: An Equivariance Module to Improve Self-Supervised Learning [77.34726150561087]
  Self-supervised visual representation methods are closing the gap with supervised learning performance. These methods rely on maximizing the similarity between embeddings of related synthetic inputs created through data augmentations. We introduce EquiMod a generic equivariance module that structures the learned latent space.
  arXiv  Detail & Related papers  (2022-11-02T16:25:54Z)
• Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation [0.0]
  Multiformer is a Transformer-based model which allows the use of different attention mechanisms on each head. By doing this, the model is able to bias the self-attention towards the extraction of more diverse token interactions. Results show that mixing attention patterns along the different heads and layers outperforms our baseline by up to 0.7 BLEU.
  arXiv  Detail & Related papers  (2022-05-14T17:37:47Z)
• Transformers with Competitive Ensembles of Independent Mechanisms [97.93090139318294]
  We propose a new Transformer layer which divides the hidden representation and parameters into multiple mechanisms, which only exchange information through attention. We study TIM on a large-scale BERT model, on the Image Transformer, and on speech enhancement and find evidence for semantically meaningful specialization as well as improved performance.
  arXiv  Detail & Related papers  (2021-02-27T21:48:46Z)
• Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation [97.22768624862111]
  We analyze NMT models which explicitly evaluates the source and target relative contributions to the generation process. We find that models trained with more data tend to rely on source information more and to have more sharp token contributions.
  arXiv  Detail & Related papers  (2020-10-21T11:37:27Z)
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention [37.111204321059084]
  We propose new pretrained contextualized representations of words and entities based on the bidirectional transformer. Our model is trained using a new pretraining task based on the masked language model of BERT. We also propose an entity-aware self-attention mechanism that is an extension of the self-attention mechanism of the transformer.
  arXiv  Detail & Related papers  (2020-10-02T15:38:03Z)
• Self-Attention Attribution: Interpreting Information Interactions Inside Transformer [89.21584915290319]
  We propose a self-attention attribution method to interpret the information interactions inside Transformer. We show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.
  arXiv  Detail & Related papers  (2020-04-23T14:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.