CrossICL: Cross-Task In-Context Learning via Unsupervised Demonstration Transfer

• URL: http://arxiv.org/abs/2505.24143v1
• Date: Fri, 30 May 2025 02:26:05 GMT
• Title: CrossICL: Cross-Task In-Context Learning via Unsupervised Demonstration Transfer
• Authors: Jinglong Gao, Xiao Ding, Lingxiao Zou, Bing Qin, Ting Liu,
• Abstract summary: In-Context Learning (ICL) enhances the performance of large language models (LLMs) with demonstrations.<n>In most real-world scenarios, users are often unwilling or unable to provide such demonstrations.<n>Inspired by the human analogy, we explore a new ICL paradigm CrossICL to study how to utilize existing source task demonstrations in the ICL for target tasks.
• Score: 31.81611723106955
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In-Context Learning (ICL) enhances the performance of large language models (LLMs) with demonstrations. However, obtaining these demonstrations primarily relies on manual effort. In most real-world scenarios, users are often unwilling or unable to provide such demonstrations. Inspired by the human analogy, we explore a new ICL paradigm CrossICL to study how to utilize existing source task demonstrations in the ICL for target tasks, thereby obtaining reliable guidance without any additional manual effort. To explore this, we first design a two-stage alignment strategy to mitigate the interference caused by gaps across tasks, as the foundation for our experimental exploration. Based on it, we conduct comprehensive exploration of CrossICL, with 875 NLP tasks from the Super-NI benchmark and six types of LLMs, including GPT-4o. Experimental results demonstrate the effectiveness of CrossICL and provide valuable insights on questions like the criteria for selecting cross-task demonstrations, as well as the types of task-gap-induced interference in CrossICL.

Related papers

• Enhancing Cross-task Transfer of Large Language Models via Activation Steering [75.41750053623298]
  Cross-task in-context learning offers a direct solution for transferring knowledge across tasks.<n>We investigate whether cross-task transfer can be achieved via latent space steering without parameter updates or input expansion.<n>We propose a novel Cross-task Activation Steering Transfer framework that enables effective transfer by manipulating the model's internal activation states.
  arXiv  Detail & Related papers  (2025-07-17T15:47:22Z)
• Leveraging In-Context Learning for Language Model Agents [51.2996117207114]
  In-context learning (ICL) with dynamically selected demonstrations combines the flexibility of prompting large language models (LLMs) with the ability to leverage training data to improve performance.<n>We show that set-selection of trajectories of similar tasks as demonstrations significantly improves performance, reliability, robustness, and efficiency of LLM agents.<n>We find that demonstrations obtained from larger models (in the annotation phase) also improve smaller models, and that ICL agents can even rival costlier trained agents.
  arXiv  Detail & Related papers  (2025-06-16T05:37:49Z)
• What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration [59.855712519568904]
  We investigate the three core steps of MM-ICL including demonstration retrieval, demonstration ordering, and prompt construction. Our findings highlight the necessity of a multi-modal retriever for demonstration retrieval, and the importance of intra-demonstration ordering over inter-demonstration ordering.
  arXiv  Detail & Related papers  (2024-10-27T15:37:51Z)
• DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning [99.05401042153214]
  We propose a Demonstration-aware Monte Carlo Tree Search (MCTS) approach (DAWN-ICL) to conduct in-context learning (ICL)<n>In real-world scenarios, problems usually come from diverse tasks, and only a few belong to the same task. The random order may generate unreliable pseudo-demonstrations and lead to error accumulation.
  arXiv  Detail & Related papers  (2024-10-26T16:17:02Z)
• In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks [93.46282380831339]
  In-context learning helps large language models adapt to various tasks by providing demonstrations of the target task. We propose In-Context Transfer Learning (ICTL), which synthesizes target task demonstrations by transferring labeled demonstrations from similar source tasks. Experiments on Super-NI show that ICTL outperforms synthesis from scratch by 2.0% on average.
  arXiv  Detail & Related papers  (2024-10-02T13:37:54Z)
• Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning [43.356895599336504]
  We analyze the working mechanisms of the learning-based demonstration selection methods. We empirically identify two important factors related to similarity measurement. We introduce two effective yet simplified exemplar selection methods catering to task-agnostic and task-specific demands.
  arXiv  Detail & Related papers  (2024-06-14T03:34:02Z)
• TEGEE: Task dEfinition Guided Expert Ensembling for Generalizable and Few-shot Learning [37.09785060896196]
  We propose textbfTEGEE (Task Definition Guided Expert Ensembling), a method that explicitly extracts task definitions.<n>Our framework employs a dual 3B model approach, with each model assigned a distinct role.<n> Empirical evaluations show that TEGEE performs comparably to the larger LLaMA2-13B model.
  arXiv  Detail & Related papers  (2024-03-07T05:26:41Z)
• Misconfidence-based Demonstration Selection for LLM In-Context Learning [0.0]
  In-context learning with large language models (LLMs) excels at adapting to various tasks rapidly. Current approaches to this problem either rely on hard-to-acquire external supervision or require frequent interactions with LLMs. We propose a new method called In-Context Reflection (ICR) to overcome these challenges.
  arXiv  Detail & Related papers  (2024-01-12T00:11:24Z)
• Comparable Demonstrations are Important in In-Context Learning: A Novel Perspective on Demonstration Selection [22.29452683679149]
  In-Context Learning (ICL) is an important paradigm for adapting Large Language Models (LLMs) to downstream tasks through a few demonstrations. This study explores the ICL mechanisms from a novel perspective, providing a deeper insight into the demonstration selection strategy for ICL.
  arXiv  Detail & Related papers  (2023-12-12T18:05:46Z)
• LLMs Learn Task Heuristics from Demonstrations: A Heuristic-Driven Prompting Strategy for Document-Level Event Argument Extraction [12.673710691468264]
  We introduce the Heuristic-Driven Link-of- Analogy (HD-LoA) prompting to address the challenge of example selection. Inspired by the analogical reasoning of human, we propose the link-of-analogy prompting, which enables LLMs to process new situations. Experiments show that our method outperforms existing prompting methods and few-shot supervised learning methods on document-level EAE datasets.
  arXiv  Detail & Related papers  (2023-11-11T12:05:01Z)
• Iterative Forward Tuning Boosts In-Context Learning in Language Models [88.25013390669845]
  In this study, we introduce a novel two-stage framework to boost in-context learning in large language models (LLMs) Specifically, our framework delineates the ICL process into two distinct stages: Deep-Thinking and test stages. The Deep-Thinking stage incorporates a unique attention mechanism, i.e., iterative enhanced attention, which enables multiple rounds of information accumulation.
  arXiv  Detail & Related papers  (2023-05-22T13:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.