A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities

• URL: http://arxiv.org/abs/2609.03687v1
• Date: Thu, 03 Sep 2026 11:23:11 GMT
• Title: A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities
• Abstract summary: We investigate the mechanism for representing and retrieving singular and plural entities for plural reference.<n>We find a set of attention heads that are responsible for representing coreference information in the input.<n>We also find that LLMs align with humans in preference for plural pronoun.
• Score: 3.409902233585822
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coreference resolution is an important task in contextual reasoning. In this paper, we investigate the mechanism for representing and retrieving singular and plural entities for plural reference. We use a combination of mechanistic interpretability and attention pattern analysis to study the process in which LLMs predict a pronoun to refer back to previously mentioned entities. Using a range of causal intervention techniques, we find a set of attention heads that are responsible for (1) representing coreference information in the input, (2) identifying entities that form a plural reference, (3) transferring the information to the component that is responsible for selecting the antecedents and predicting the pronoun. We also find that LLMs align with humans in preference for plural pronoun. Specifically, entities in a plural construction are more likely to be referred to as a plural entity if they are ontologically similar and are linked by the conjunction "and".

Related papers

• Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference [3.409902233585822]
  LLMs are sometimes aware of possible referents of ambiguous pronouns.<n>They do not always follow human reference when choosing between interpretations.<n>They struggle to identify ambiguity without direct instruction.
  arXiv  Detail & Related papers  (2025-10-06T08:32:59Z)
• Evaluating Large Language Models as Expert Annotators [17.06186816803593]
  This paper investigates whether top-performing language models can serve as direct alternatives to human expert annotators.<n>We evaluate both individual LLMs and multi-agent approaches across three highly specialized domains: finance, biomedicine, and law.<n>Our empirical results reveal that individual LLMs equipped with inference-time techniques show only marginal or even negative performance gains.
  arXiv  Detail & Related papers  (2025-08-11T10:19:10Z)
• Language Models use Lookbacks to Track Beliefs [53.8367373163602]
  We analyze LMs' ability to reason about characters' beliefs using causal mediation and abstraction.<n>Our work provides insights into belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs.
  arXiv  Detail & Related papers  (2025-05-20T17:59:45Z)
• A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition [71.61103962200666]
  Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora.<n>Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates.<n>We introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER.
  arXiv  Detail & Related papers  (2025-02-25T23:30:43Z)
• How Do Multilingual Language Models Remember Facts? [50.13632788453612]
  We show that previously identified recall mechanisms in English largely apply to multilingual contexts.<n>We localize the role of language during recall, finding that subject enrichment is language-independent.<n>In decoder-only LLMs, FVs compose these two pieces of information in two separate stages.
  arXiv  Detail & Related papers  (2024-10-18T11:39:34Z)
• Analyzing the Role of Semantic Representations in the Era of Large Language Models [104.18157036880287]
  We investigate the role of semantic representations in the era of large language models (LLMs) We propose an AMR-driven chain-of-thought prompting method, which we call AMRCoT. We find that it is difficult to predict which input examples AMR may help or hurt on, but errors tend to arise with multi-word expressions.
  arXiv  Detail & Related papers  (2024-05-02T17:32:59Z)
• A Roadmap to Pluralistic Alignment [49.29107308098236]
  We propose a roadmap to pluralistic alignment, specifically using language models as a test bed. We identify and formalize three possible ways to define and operationalize pluralism in AI systems. We argue that current alignment techniques may be fundamentally limited for pluralistic AI.
  arXiv  Detail & Related papers  (2024-02-07T18:21:17Z)
• Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models [91.3755431537592]
  Representing compositional and non-compositional phrases is critical for language understanding. We first formulate a problem of predicting the LM-internal representations of longer phrases given those of their constituents. While we would expect the predictive accuracy to correlate with human judgments of semantic compositionality, we find this is largely not the case.
  arXiv  Detail & Related papers  (2022-10-07T14:21:30Z)
• Making sense of spoken plurals [1.80476943513092]
  This study focuses on the semantics of noun singulars and their plural inflectional variants in English. One model (FRACSS) proposes that all singular-plural pairs should be taken into account when predicting plural semantics from singular semantics. The other model (CCA) argues that conceptualization for plurality depends primarily on the semantic class of the base word.
  arXiv  Detail & Related papers  (2022-07-05T10:44:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.