To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

• URL: http://arxiv.org/abs/2601.12075v1
• Date: Sat, 17 Jan 2026 14:46:29 GMT
• Title: To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
• Authors: Mehrdad Farahani, Franziska Penzkofer, Richard Johansson,
• Abstract summary: Language models must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt.<n>This work presents a mechanistic study of that choice by extracting an empharbitration vector from model activations on a curated dataset.
• Score: 5.057026826740146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an \emph{arbitration vector} from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: Copy$\rightarrow$Recall (suppressing context use) and Recall$\rightarrow$Copy (inducing the model to copy any token from the context). Experiments on two architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy ``reactivation'' process that can be triggered at different locations in the input, while restoring recall is a ``suppression'' process that is more fragile and strongly tied to object-token interventions.

Related papers

• Gaussian Match-and-Copy: A Minimalist Benchmark for Studying Transformer Induction [44.83333974000826]
  We introduce a minimalist benchmark that isolates long-range retrieval through pure second-order correlation signals.<n> Numerical investigations show that this task retains key qualitative aspects of how Transformers develop match-and-copy circuits.<n>We prove this max-margin alignment for GD trajectories that reach vanishing empirical loss under explicit technical conditions.
  arXiv  Detail & Related papers  (2026-02-07T14:18:11Z)
• Smoothing the Black-Box: Signed-Distance Supervision for Black-Box Model Copying [0.6015898117103069]
  Black-box copying provides a practical mechanism to upgrade legacy models.<n>When restricted to hard-label outputs, copying turns into a discontinuous surface reconstruction problem.<n>We propose a distance-based copying framework that replaces hard-label supervision with signed distances to the teacher's decision boundary.
  arXiv  Detail & Related papers  (2026-01-28T17:00:04Z)
• MSN: Multi-directional Similarity Network for Hand-crafted and Deep-synthesized Copy-Move Forgery Detection [41.87843079741093]
  We propose a novel two-stream model, namely Multi-directional Similarity Network (MSN), to accurate and efficient copy-move forgery detection.<n>In representation, an image is hierarchically encoded by a multi-directional CNN network, and due to the diverse augmentation in scales and rotations, the feature achieved better measures the similarity between sampled patches in two streams.<n>In localization, we design a 2-D similarity matrix based decoder, and compared with the current 1-D similarity vector based one, it makes full use of spatial information in the entire image.
  arXiv  Detail & Related papers  (2025-12-08T02:47:05Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Copy-Paste to Mitigate Large Language Model Hallucinations [28.490445724463864]
  We propose CopyPasteLLM, obtained through two-stage high-copying response preference training.<n>On FaithEval, ConFiQA and PubMedQA, CopyPasteLLM achieves best performance in both counterfactual and original contexts.<n>To elucidate CopyPasteLLM's effectiveness, we propose the Context- Copying Capturing algorithm.
  arXiv  Detail & Related papers  (2025-10-01T04:40:04Z)
• CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation [132.00910067533982]
  We introduce CopyBench, a benchmark designed to measure both literal and non-literal copying in LM generations. We find that, although literal copying is relatively rare, two types of non-literal copying -- event copying and character copying -- occur even in models as small as 7B parameters.
  arXiv  Detail & Related papers  (2024-07-09T17:58:18Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding [34.078590816368056]
  We study the problem of visual grounding by considering both phrase extraction and grounding (PEG) PEG requires a model to extract phrases from text and locate objects from images simultaneously. We propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text.
  arXiv  Detail & Related papers  (2022-11-28T16:30:46Z)
• Hybrid Routing Transformer for Zero-Shot Learning [83.64532548391]
  This paper presents a novel transformer encoder-decoder model, called hybrid routing transformer (HRT) We embed an active attention, which is constructed by both the bottom-up and the top-down dynamic routing pathways to generate the attribute-aligned visual feature. While in HRT decoder, we use static routing to calculate the correlation among the attribute-aligned visual features, the corresponding attribute semantics, and the class attribute vectors to generate the final class label predictions.
  arXiv  Detail & Related papers  (2022-03-29T07:55:08Z)
• Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detection [0.0]
  Reconstruction autoencoder-based methods deal with the problem by using input reconstruction error as a metric of novelty vs. normality. We introduce semantic reconstruction, data certainty decomposition and normalized L2 distance to substantially improve original methods. Our method works without any additional data, hard-to-implement structure, time-consuming pipeline, and even harming the classification accuracy of known classes.
  arXiv  Detail & Related papers  (2022-03-04T09:04:55Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)
• Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder [104.25716317141321]
  We propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach provides state-of-the-art performance on both Event2Mind and ATOMIC datasets.
  arXiv  Detail & Related papers  (2020-06-15T02:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.