XPrompt:Explaining Large Language Model's Generation via Joint Prompt Attribution

• URL: http://arxiv.org/abs/2405.20404v1
• Date: Thu, 30 May 2024 18:16:41 GMT
• Title: XPrompt:Explaining Large Language Model's Generation via Joint Prompt Attribution
• Authors: Yurui Chang, Bochuan Cao, Yujia Wang, Jinghui Chen, Lu Lin,
• Abstract summary: Large Language Models (LLMs) have demonstrated impressive performances in complex text generation tasks. The contribution of the input prompt to the generated content still remains obscure to humans. We introduce a counterfactual explanation framework based on joint prompt attribution, XPrompt.
• Score: 26.639271355209104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have demonstrated impressive performances in complex text generation tasks. However, the contribution of the input prompt to the generated content still remains obscure to humans, underscoring the necessity of elucidating and explaining the causality between input and output pairs. Existing works for providing prompt-specific explanation often confine model output to be classification or next-word prediction. Few initial attempts aiming to explain the entire language generation often treat input prompt texts independently, ignoring their combinatorial effects on the follow-up generation. In this study, we introduce a counterfactual explanation framework based on joint prompt attribution, XPrompt, which aims to explain how a few prompt texts collaboratively influences the LLM's complete generation. Particularly, we formulate the task of prompt attribution for generation interpretation as a combinatorial optimization problem, and introduce a probabilistic algorithm to search for the casual input combination in the discrete space. We define and utilize multiple metrics to evaluate the produced explanations, demonstrating both faithfulness and efficiency of our framework.

Related papers

• Likelihood as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that likelihoods serve as an effective gauge for language model performance. We propose two methods that use question likelihood as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• Graph-DPEP: Decomposed Plug and Ensemble Play for Few-Shot Document Relation Extraction with Graph-of-Thoughts Reasoning [34.85741925091139]
  Graph-DPEP framework is grounded in the reasoning behind triplet explanation thoughts presented in natural language. We develop "ensemble-play", reapplying generation on the entire type list by leveraging the reasoning thoughts embedded in a sub-graph.
  arXiv  Detail & Related papers  (2024-11-05T07:12:36Z)
• PromptExp: Multi-granularity Prompt Explanation of Large Language Models [16.259208045898415]
  We introduce PromptExp, a framework for multi-granularity prompt explanations by aggregating token-level insights. PromptExp supports both white-box and black-box explanations and extends explanations to higher granularity levels. We evaluate PromptExp in case studies such as sentiment analysis, showing the perturbation-based approach performs best.
  arXiv  Detail & Related papers  (2024-10-16T22:25:15Z)
• Towards LLM-guided Causal Explainability for Black-box Text Classifiers [16.36602400590088]
  We aim to leverage the instruction-following and textual understanding capabilities of recent Large Language Models to facilitate causal explainability. We propose a three-step pipeline via which, we use an off-the-shelf LLM to identify the latent or unobserved features in the input text. We experiment with our pipeline on multiple NLP text classification datasets, and present interesting and promising findings.
  arXiv  Detail & Related papers  (2023-09-23T11:22:28Z)
• On the Role of Attention in Prompt-tuning [90.97555030446563]
  We study prompt-tuning for one-layer attention architectures and study contextual mixture-models. We show that softmax-prompt-attention is provably more expressive than softmax-self-attention and linear-prompt-attention. We also provide experiments that verify our theoretical insights on real datasets and demonstrate how prompt-tuning enables the model to attend to context-relevant information.
  arXiv  Detail & Related papers  (2023-06-06T06:23:38Z)
• Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios [18.5713713816771]
  This paper is the first to explore the problem smoothly from weak-supervised learning to unsupervised learning. We propose a non-autoregressive interpretable model to facilitate parallel explanation generation and simultaneous prediction.
  arXiv  Detail & Related papers  (2023-02-21T14:52:21Z)
• Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
  Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
  arXiv  Detail & Related papers  (2023-02-09T18:02:34Z)
• Explaining Patterns in Data with Language Models via Interpretable Autoprompting [143.4162028260874]
  We introduce interpretable autoprompting (iPrompt), an algorithm that generates a natural-language string explaining the data. iPrompt can yield meaningful insights by accurately finding groundtruth dataset descriptions. Experiments with an fMRI dataset show the potential for iPrompt to aid in scientific discovery.
  arXiv  Detail & Related papers  (2022-10-04T18:32:14Z)
• Local Explanation of Dialogue Response Generation [77.68077106724522]
  Local explanation of response generation (LERG) is proposed to gain insights into the reasoning process of a generation model. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. Our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%.
  arXiv  Detail & Related papers  (2021-06-11T17:58:36Z)
• Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection [62.071938098215085]
  We focus on the Commongen benchmark, wherein the aim is to generate a plausible sentence for a given set of input concepts. We propose strategies for enhancing the semantic correctness of the generated text.
  arXiv  Detail & Related papers  (2020-12-19T23:23:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.