ContextCite: Attributing Model Generation to Context
- URL: http://arxiv.org/abs/2409.00729v2
- Date: Fri, 13 Sep 2024 20:26:40 GMT
- Title: ContextCite: Attributing Model Generation to Context
- Authors: Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, Aleksander Madry,
- Abstract summary: We introduce the problem of context attribution, pinpointing the parts of the context that led a model to generate a particular statement.
We then present ContextCite, a simple and scalable method for context attribution that can be applied on top of any existing language model.
We showcase ContextCite through three applications: helping verify generated statements, improving response quality, and detecting poisoning attacks.
- Score: 64.90535024385305
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: How do language models use information provided as context when generating a response? Can we infer whether a particular generated statement is actually grounded in the context, a misinterpretation, or fabricated? To help answer these questions, we introduce the problem of context attribution: pinpointing the parts of the context (if any) that led a model to generate a particular statement. We then present ContextCite, a simple and scalable method for context attribution that can be applied on top of any existing language model. Finally, we showcase the utility of ContextCite through three applications: (1) helping verify generated statements (2) improving response quality by pruning the context and (3) detecting poisoning attacks. We provide code for ContextCite at https://github.com/MadryLab/context-cite.
Related papers
- Controllable Context Sensitivity and the Knob Behind It [53.70327066130381]
When making predictions, a language model must trade off how much it relies on its context vs. its prior knowledge.
We search for a knob which controls this sensitivity, determining whether language models answer from the context or their prior knowledge.
arXiv Detail & Related papers (2024-11-11T22:22:21Z) - Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding [9.2433070542025]
Large language models (LLMs) tend to inadequately integrate input context during text generation.
We introduce a novel approach integrating contrastive decoding with adversarial irrelevant passages as negative samples.
arXiv Detail & Related papers (2024-05-04T20:38:41Z) - Out of Context: How important is Local Context in Neural Program Repair? [5.732727528813227]
We study the importance of this local context on repair success.
We train and evaluate Transformer models in many different local context configurations.
Our results are not only relevant for researchers working on Transformer-based APR tools but also for benchmark and dataset creators.
arXiv Detail & Related papers (2023-12-08T11:49:02Z) - Context Diffusion: In-Context Aware Image Generation [29.281927418777624]
Context Diffusion is a diffusion-based framework that enables image generation models to learn from visual examples presented in context.
Our experiments and user study demonstrate that Context Diffusion excels in both in-domain and out-of-domain tasks.
arXiv Detail & Related papers (2023-12-06T16:19:51Z) - Learning to Filter Context for Retrieval-Augmented Generation [75.18946584853316]
Generation models are required to generate outputs given partially or entirely irrelevant passages.
FILCO identifies useful context based on lexical and information-theoretic approaches.
It trains context filtering models that can filter retrieved contexts at test time.
arXiv Detail & Related papers (2023-11-14T18:41:54Z) - Constructing Highly Inductive Contexts for Dialogue Safety through
Controllable Reverse Generation [65.48908724440047]
We propose a method called emphreverse generation to construct adversarial contexts conditioned on a given response.
We test three popular pretrained dialogue models (Blender, DialoGPT, and Plato2) and find that BAD+ can largely expose their safety problems.
arXiv Detail & Related papers (2022-12-04T12:23:41Z) - Context-LGM: Leveraging Object-Context Relation for Context-Aware Object
Recognition [48.5398871460388]
We propose a novel Contextual Latent Generative Model (Context-LGM), which considers the object-context relation and models it in a hierarchical manner.
To infer contextual features, we reformulate the objective function of Variational Auto-Encoder (VAE), where contextual features are learned as a posterior conditioned distribution on the object.
The effectiveness of our method is verified by state-of-the-art performance on two context-aware object recognition tasks.
arXiv Detail & Related papers (2021-10-08T11:31:58Z) - Do Context-Aware Translation Models Pay the Right Attention? [61.25804242929533]
Context-aware machine translation models are designed to leverage contextual information, but often fail to do so.
In this paper, we ask several questions: What contexts do human translators use to resolve ambiguous words?
We introduce SCAT (Supporting Context for Ambiguous Translations), a new English-French dataset comprising supporting context words for 14K translations.
Using SCAT, we perform an in-depth analysis of the context used to disambiguate, examining positional and lexical characteristics of the supporting words.
arXiv Detail & Related papers (2021-05-14T17:32:24Z) - Decontextualization: Making Sentences Stand-Alone [13.465459751619818]
Models for question answering, dialogue agents, and summarization often interpret the meaning of a sentence in a rich context.
Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window.
We define the problem of sentence decontextualization: taking a sentence together with its context and rewriting it to be interpretable out of context.
arXiv Detail & Related papers (2021-02-09T22:52:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.