How do Language Models Bind Entities in Context?

• URL: http://arxiv.org/abs/2310.17191v2
• Date: Mon, 6 May 2024 10:09:48 GMT
• Title: How do Language Models Bind Entities in Context?
• Abstract summary: Language models (LMs) must bind entities to their attributes to correctly use in-context information. We identify the binding ID mechanism: a general mechanism for solving the binding problem. Our results uncover interpretable strategies in LMs for representing symbolic knowledge in-context.
• Score: 31.529253469951062
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: To correctly use in-context information, language models (LMs) must bind entities to their attributes. For example, given a context describing a "green square" and a "blue circle", LMs must bind the shapes to their respective colors. We analyze LM representations and identify the binding ID mechanism: a general mechanism for solving the binding problem, which we observe in every sufficiently large model from the Pythia and LLaMA families. Using causal interventions, we show that LMs' internal activations represent binding information by attaching binding ID vectors to corresponding entities and attributes. We further show that binding ID vectors form a continuous subspace, in which distances between binding ID vectors reflect their discernability. Overall, our results uncover interpretable strategies in LMs for representing symbolic knowledge in-context, providing a step towards understanding general in-context reasoning in large-scale LMs.

Related papers

• ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships [53.24877477312131]
  ReBind is a systematic framework that introduces semantic instructions with embedded reference tokens.<n>We develop ReBind-Instruct, a specialized MLLM that learns to establish explicit bindings between visual attributes and their reference sources.<n>We also develop ReBind-Edit, which enables lightweight adaptation of text-to-video models to coordinate multiple references.
  arXiv  Detail & Related papers  (2026-07-16T07:43:27Z)
• A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models [16.759528230678722]
  We identify a retrieval conditioned rebinding mechanism that encodes swap relevant binding information and reinstates it at readout.<n>Across Gemma and Llama models, this circuit supports rebinding behavior, but the representational signature of the mechanism differs across model families.
  arXiv  Detail & Related papers  (2026-06-07T14:16:44Z)
• On the Representations of Entities in Auto-regressive Large Language Models [10.08529318994409]
  We introduce entity mention reconstruction as a novel framework for studying how Large Language Models (LLMs) encode and manipulate entities.<n>We investigate whether entity mentions can be generated from internal representations, how multi-token entities are encoded beyond last-token embeddings, and whether these representations capture relational knowledge.
  arXiv  Detail & Related papers  (2025-10-10T14:23:44Z)
• Visual symbolic mechanisms: Emergent symbol processing in vision language models [1.0923877073891446]
  We identify a set of emergent symbolic mechanisms that support binding in vision language models (VLMs)<n>We find that binding errors can be traced directly to failures in these mechanisms.<n> Taken together, these results shed light on the mechanisms that support symbol-like processing in VLMs, and suggest possible avenues for addressing the persistent binding failures exhibited by these models.
  arXiv  Detail & Related papers  (2025-06-18T20:35:44Z)
• On Entity Identification in Language Models [22.988199295121134]
  Internal representations of language models (LMs) identify and distinguish mentions of named entities.<n>We first formulate two problems of entity mentions -- ambiguity and variability.<n>We propose a framework analogous to clustering quality metrics.
  arXiv  Detail & Related papers  (2025-06-03T09:55:21Z)
• Investigating Mechanisms for In-Context Vision Language Binding [17.440725558041258]
  Vision-Language models (VLMs) must perceive the image, comprehend the text, and build associations within and across both modalities.<n>For instance, given an 'image of a red toy car', the model should associate this image to phrases like 'car','red toy','red object', etc.
  arXiv  Detail & Related papers  (2025-05-28T10:25:43Z)
• Language Models use Lookbacks to Track Beliefs [51.92689607136242]
  We analyze Llama-3-70B-Instruct's ability to reason about characters' beliefs using causal mediation and abstraction.<n>Our work provides insights into the LM's belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs.
  arXiv  Detail & Related papers  (2025-05-20T17:59:45Z)
• The Representation and Recall of Interwoven Structured Knowledge in LLMs: A Geometric and Layered Analysis [0.0]
  Large language models (LLMs) represent and recall multi-associated attributes across transformer layers. intermediate layers encode factual knowledge by superimposing related attributes in overlapping spaces. later layers refine linguistic patterns and progressively separate attribute representations.
  arXiv  Detail & Related papers  (2025-02-15T18:08:51Z)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• Core Context Aware Transformers for Long Context Language Modeling [50.774702091154204]
  We propose a plug-and-play Core Context Aware (CCA) Attention for efficient long-context modeling.<n>Our method automatically focuses and strengthens core context while diminishing redundancy during the learning process.<n>Our method is able to replace the self-attention module in existing Large Language Models with minimal fine-tuning cost.
  arXiv  Detail & Related papers  (2024-12-17T01:54:08Z)
• Representational Analysis of Binding in Language Models [21.567939573751254]
  Language models (LMs) must bind an entity to its attribute to recall attribute for a given entity. We show that there exists a low-rank subspace in the activations of LMs, that primarily encodes the order (i.e., OI) of entity and attribute. We also discover the causal effect of OI on binding that when editing representations along the OI encoding direction, LMs tend to bind a given entity to other attributes accordingly.
  arXiv  Detail & Related papers  (2024-09-09T09:04:56Z)
• ClawMachine: Fetching Visual Tokens as An Entity for Referring and Grounding [67.63933036920012]
  Existing methods, including proxy encoding and geometry encoding, incorporate additional syntax to encode the object's location. This study presents ClawMachine, offering a new methodology that notates an entity directly using the visual tokens. ClawMachine unifies visual referring and grounding into an auto-regressive format and learns with a decoder-only architecture.
  arXiv  Detail & Related papers  (2024-06-17T08:39:16Z)
• Understanding Information Storage and Transfer in Multi-modal Large Language Models [51.20840103605018]
  We study how Multi-modal Large Language Models process information in a factual visual question answering task. Key findings show that these MLLMs rely on self-attention blocks in much earlier layers for information storage. We introduce MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs.
  arXiv  Detail & Related papers  (2024-06-06T16:35:36Z)
• Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering [9.86691461253151]
  We introduce a novel method for attribution in contextual question answering, leveraging the hidden state representations of large language models (LLMs) Our approach bypasses the need for extensive model retraining and retrieval model overhead, offering granular attributions and preserving the quality of generated answers. We present Verifiability-granular, an attribution dataset which has token level annotations for LLM generations in the contextual question answering setup.
  arXiv  Detail & Related papers  (2024-05-28T09:12:44Z)
• RAGE Against the Machine: Retrieval-Augmented LLM Explanations [17.707251978644486]
  RAGE is an interactive tool for explaining Large Language Models (LLMs) Our explanations are counterfactual in the sense that they identify parts of the input context that, when removed, change the answer to the question posed to the LLM. RAGE includes pruning methods to navigate the vast space of possible explanations, allowing users to view the provenance of the produced answers.
  arXiv  Detail & Related papers  (2024-05-11T19:08:38Z)
• Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment [31.70064035432789]
  We propose a Large Language Model-enhanced Entity Alignment framework (LLMEA) LLMEA identifies candidate alignments for a given entity by considering both embedding similarities between entities across Knowledge Graphs and edit distances to a virtual equivalent entity. Experiments conducted on three public datasets reveal that LLMEA surpasses leading baseline models.
  arXiv  Detail & Related papers  (2024-01-30T12:41:04Z)
• Exploiting Contextual Target Attributes for Target Sentiment Classification [53.30511968323911]
  Existing PTLM-based models for TSC can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. We present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes.
  arXiv  Detail & Related papers  (2023-12-21T11:45:28Z)
• Word Embeddings Are Steers for Language Models [57.83026781380927]
  We name such steers LM-Steers and find them existing in LMs of all sizes. On tasks such as language model detoxification and sentiment control, LM-Steers can achieve comparable or superior performance. An LM-Steer is transferrable between different language models by an explicit form calculation.
  arXiv  Detail & Related papers  (2023-05-22T07:52:04Z)
• Empowering Language Models with Knowledge Graph Reasoning for Question Answering [117.79170629640525]
  We propose knOwledge REasOning empowered Language Model (OREO-LM) OREO-LM consists of a novel Knowledge Interaction Layer that can be flexibly plugged into existing Transformer-based LMs. We show significant performance gain, achieving state-of-art results in the Closed-Book setting.
  arXiv  Detail & Related papers  (2022-11-15T18:26:26Z)
• Open Relation Modeling: Learning to Define Relations between Entities [24.04238065663009]
  We propose to teach machines to generate definition-like relation descriptions by letting them learn from definitions of entities. Specifically, we fine-tune Pre-trained Language Models (PLMs) to produce definitions conditioned on extracted entity pairs. We show that PLMs can select interpretable and informative reasoning paths by confidence estimation, and the selected path can guide PLMs to generate better relation descriptions.
  arXiv  Detail & Related papers  (2021-08-20T16:03:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.