How are Prompts Different in Terms of Sensitivity?

• URL: http://arxiv.org/abs/2311.07230v2
• Date: Thu, 20 Jun 2024 22:41:35 GMT
• Title: How are Prompts Different in Terms of Sensitivity?
• Authors: Sheng Lu, Hendrik Schuff, Iryna Gurevych,
• Abstract summary: We present a comprehensive prompt analysis based on the sensitivity of a function. We use gradient-based saliency scores to empirically demonstrate how different prompts affect the relevance of input tokens to the output. We introduce sensitivity-aware decoding which incorporates sensitivity estimation as a penalty term in the standard greedy decoding.
• Score: 50.67313477651395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-context learning (ICL) has become one of the most popular learning paradigms. While there is a growing body of literature focusing on prompt engineering, there is a lack of systematic analysis comparing the effects of prompts across different models and tasks. To address this gap, we present a comprehensive prompt analysis based on the sensitivity of a function. Our analysis reveals that sensitivity is an unsupervised proxy for model performance, as it exhibits a strong negative correlation with accuracy. We use gradient-based saliency scores to empirically demonstrate how different prompts affect the relevance of input tokens to the output, resulting in different levels of sensitivity. Furthermore, we introduce sensitivity-aware decoding which incorporates sensitivity estimation as a penalty term in the standard greedy decoding. We show that this approach is particularly helpful when information in the input is scarce. Our work provides a fresh perspective on the analysis of prompts, and contributes to a better understanding of the mechanism of ICL.

Related papers

• ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs [72.13489820420726]
  ProSA is a framework designed to evaluate and comprehend prompt sensitivity in large language models. Our study uncovers that prompt sensitivity fluctuates across datasets and models, with larger models exhibiting enhanced robustness.
  arXiv  Detail & Related papers  (2024-10-16T09:38:13Z)
• Validation, Robustness, and Accuracy of Perturbation-Based Sensitivity Analysis Methods for Time-Series Deep Learning Models [0.0]
  This work undertakes studies to evaluate Interpretability Methods for Time-Series Deep Learning. My work will investigate perturbation-based sensitivity Analysis methods on modern Transformer models to benchmark their performances.
  arXiv  Detail & Related papers  (2024-01-29T19:51:50Z)
• Automatic Sensor-free Affect Detection: A Systematic Literature Review [0.0]
  This paper provides a comprehensive literature review on sensor-free affect detection. Despite the field's evident maturity, demonstrated by the consistent performance of the models, there is ample scope for future research. There is also a need to refine model development practices and methods.
  arXiv  Detail & Related papers  (2023-10-11T13:24:27Z)
• RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search [51.09723403468361]
  We propose a Relation and Sensitivity aware representation learning method (RaSa) RaSa includes two novel tasks: Relation-Aware learning (RA) and Sensitivity-Aware learning (SA) Experiments demonstrate that RaSa outperforms existing state-of-the-art methods by 6.94%, 4.45% and 15.35% in terms of Rank@1 on datasets.
  arXiv  Detail & Related papers  (2023-05-23T03:53:57Z)
• Language Model Classifier Aligns Better with Physician Word Sensitivity than XGBoost on Readmission Prediction [86.15787587540132]
  We introduce sensitivity score, a metric that scrutinizes models' behaviors at the vocabulary level. Our experiments compare the decision-making logic of clinicians and classifiers based on rank correlations of sensitivity scores.
  arXiv  Detail & Related papers  (2022-11-13T23:59:11Z)
• On the Relation between Sensitivity and Accuracy in In-context Learning [41.27837171531926]
  In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios. We study the sensitivity of ICL with respect to multiple perturbation types. We propose textscSenSel, a few-shot selective prediction method that abstains from sensitive predictions.
  arXiv  Detail & Related papers  (2022-09-16T00:52:34Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Balancing Robustness and Sensitivity using Feature Contrastive Learning [95.86909855412601]
  Methods that promote robustness can hurt the model's sensitivity to rare or underrepresented patterns. We propose Feature Contrastive Learning (FCL) that encourages a model to be more sensitive to the features that have higher contextual utility.
  arXiv  Detail & Related papers  (2021-05-19T20:53:02Z)
• Sensitivity as a Complexity Measure for Sequence Classification Tasks [24.246784593571626]
  We argue that standard sequence classification methods are biased towards learning low-sensitivity functions, so that tasks requiring high sensitivity are more difficult. We estimate sensitivity on 15 NLP tasks, finding that sensitivity is higher on challenging tasks collected in GLUE than on simple text classification tasks.
  arXiv  Detail & Related papers  (2021-04-21T03:56:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.