The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis

• URL: http://arxiv.org/abs/2311.00237v3
• Date: Thu, 03 Oct 2024 17:25:02 GMT
• Title: The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis
• Authors: Yuxiang Zhou, Jiazheng Li, Yanzheng Xiang, Hanqi Yan, Lin Gui, Yulan He,
• Abstract summary: In-context learning (ICL) capability enables large language models to excel in proficiency through demonstration examples. In this paper, we present a thorough survey on the interpretation and analysis of in-context learning. We believe that our work establishes the basis for further exploration into the interpretation of in-context learning.
• Score: 20.142154624977582
• License:
• Abstract: Understanding in-context learning (ICL) capability that enables large language models (LLMs) to excel in proficiency through demonstration examples is of utmost importance. This importance stems not only from the better utilization of this capability across various tasks, but also from the proactive identification and mitigation of potential risks, including concerns regarding truthfulness, bias, and toxicity, that may arise alongside the capability. In this paper, we present a thorough survey on the interpretation and analysis of in-context learning. First, we provide a concise introduction to the background and definition of in-context learning. Then, we give an overview of advancements from two perspectives: 1) a theoretical perspective, emphasizing studies on mechanistic interpretability and delving into the mathematical foundations behind ICL; and 2) an empirical perspective, concerning studies that empirically analyze factors associated with ICL. We conclude by highlighting the challenges encountered and suggesting potential avenues for future research. We believe that our work establishes the basis for further exploration into the interpretation of in-context learning. Additionally, we have created a repository containing the resources referenced in our survey.

Related papers

• Investigating Expert-in-the-Loop LLM Discourse Patterns for Ancient Intertextual Analysis [0.0]
  The study demonstrates that large language models can detect direct quotations, allusions, and echoes between texts. The model struggles with long query passages and the inclusion of false intertextual dependences. The expert-in-the-loop methodology presented offers a scalable approach for intertextual research.
  arXiv  Detail & Related papers  (2024-09-03T13:23:11Z)
• Large Language Models are Limited in Out-of-Context Knowledge Reasoning [65.72847298578071]
  Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. This paper focuses on a significant aspect of out-of-context reasoning: Out-of-Context Knowledge Reasoning (OCKR), which is to combine multiple knowledge to infer new knowledge.
  arXiv  Detail & Related papers  (2024-06-11T15:58:59Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Can Large Language Models Understand Context? [17.196362853457412]
  This paper introduces a context understanding benchmark by adapting existing datasets to suit the evaluation of generative models. Experimental results indicate that pre-trained dense models struggle with understanding more nuanced contextual features when compared to state-of-the-art fine-tuned models. As LLM compression holds growing significance in both research and real-world applications, we assess the context understanding of quantized models under in-context-learning settings.
  arXiv  Detail & Related papers  (2024-02-01T18:55:29Z)
• From Understanding to Utilization: A Survey on Explainability for Large Language Models [27.295767173801426]
  This survey underscores the imperative for increased explainability in Large Language Models (LLMs) Our focus is primarily on pre-trained Transformer-based LLMs, which pose distinctive interpretability challenges due to their scale and complexity. When considering the utilization of explainability, we explore several compelling methods that concentrate on model editing, control generation, and model enhancement.
  arXiv  Detail & Related papers  (2024-01-23T16:09:53Z)
• Understanding In-Context Learning from Repetitions [21.28694573253979]
  This paper explores the elusive mechanism underpinning in-context learning in Large Language Models (LLMs) We quantitatively investigate the role of surface features in text generation, and empirically establish the existence of emphtoken co-occurrence reinforcement By investigating the dual impacts of these features, our research illuminates the internal workings of in-context learning and expounds on the reasons for its failures.
  arXiv  Detail & Related papers  (2023-09-30T08:13:49Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)
• A Survey on In-context Learning [77.78614055956365]
  In-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP) We first present a formal definition of ICL and clarify its correlation to related studies. We then organize and discuss advanced techniques, including training strategies, prompt designing strategies, and related analysis.
  arXiv  Detail & Related papers  (2022-12-31T15:57:09Z)
• A Survey on Interpretable Reinforcement Learning [28.869513255570077]
  This survey provides an overview of various approaches to achieve higher interpretability in reinforcement learning (RL) We distinguish interpretability (as a property of a model) and explainability (as a post-hoc operation, with the intervention of a proxy) We argue that interpretable RL may embrace different facets: interpretable inputs, interpretable (transition/reward) models, and interpretable decision-making.
  arXiv  Detail & Related papers  (2021-12-24T17:26:57Z)
• Which Mutual-Information Representation Learning Objectives are Sufficient for Control? [80.2534918595143]
  Mutual information provides an appealing formalism for learning representations of data. This paper formalizes the sufficiency of a state representation for learning and representing the optimal policy. Surprisingly, we find that two of these objectives can yield insufficient representations given mild and common assumptions on the structure of the MDP.
  arXiv  Detail & Related papers  (2021-06-14T10:12:34Z)
• AR-LSAT: Investigating Analytical Reasoning of Text [57.1542673852013]
  We study the challenge of analytical reasoning of text and introduce a new dataset consisting of questions from the Law School Admission Test from 1991 to 2016. We analyze what knowledge understanding and reasoning abilities are required to do well on this task.
  arXiv  Detail & Related papers  (2021-04-14T02:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.