From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning

• URL: http://arxiv.org/abs/2510.24528v1
• Date: Tue, 28 Oct 2025 15:37:51 GMT
• Title: From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning
• Authors: Zihan Chen, Song Wang, Xingbo Fu, Chengshuai Shi, Zhenyu Lei, Cong Shen, Jundong Li,
• Abstract summary: In-context learning (ICL) enables large language models to perform novel tasks without parameter updates.<n>We propose a cost-efficient two-stage pipeline that reduces reliance on language models for data labeling.<n> Experiments across five tasks demonstrate that our method achieves strong performance while lowering labeling costs.
• Score: 55.90498988440303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The capability of in-context learning (ICL) enables large language models (LLMs) to perform novel tasks without parameter updates by conditioning on a few input-output examples. However, collecting high-quality examples for new or challenging tasks can be costly and labor-intensive. In this work, we propose a cost-efficient two-stage pipeline that reduces reliance on LLMs for data labeling. Our approach first leverages readily available cross-task examples to prompt an LLM and pseudo-label a small set of target task instances. We then introduce a graph-based label propagation method that spreads label information to the remaining target examples without additional LLM queries. The resulting fully pseudo-labeled dataset is used to construct in-task demonstrations for ICL. This pipeline combines the flexibility of cross-task supervision with the scalability of LLM-free propagation. Experiments across five tasks demonstrate that our method achieves strong performance while lowering labeling costs.

Related papers

• TDR: Task-Decoupled Retrieval with Fine-Grained LLM Feedback for In-Context Learning [15.674990209737182]
  In-context learning (ICL) has become a classic approach for enabling LLMs to handle various tasks based on a few input-output examples.<n>The effectiveness of ICL heavily relies on the quality of these examples, and previous works which focused on enhancing example retrieval capabilities have achieved impressive performances.
  arXiv  Detail & Related papers  (2025-07-24T12:12:04Z)
• MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning [53.02571749383208]
  In-Context Learning (ICL) empowers Large Language Models (LLMs) to tackle diverse tasks by incorporating multiple input-output examples.<n>Many-Shot Adaptive Pseudo-LabEling (MAPLE) is a novel influence-based many-shot ICL framework that utilizes pseudo-labeled samples to compensate for the lack of label information.
  arXiv  Detail & Related papers  (2025-05-22T04:54:27Z)
• Augmenting In-Context-Learning in LLMs via Automatic Data Labeling and Refinement [7.108002571622824]
  We propose Automatic Data Labeling and Refinement (ADLR) to automatically generate and filter demonstrations. We demonstrate the advantage of ADLR in code-based table QA and mathematical reasoning, achieving up to a 5.5% gain.
  arXiv  Detail & Related papers  (2024-10-14T10:06:58Z)
• In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks [93.46282380831339]
  In-context learning helps large language models adapt to various tasks by providing demonstrations of the target task. We propose In-Context Transfer Learning (ICTL), which synthesizes target task demonstrations by transferring labeled demonstrations from similar source tasks. Experiments on Super-NI show that ICTL outperforms synthesis from scratch by 2.0% on average.
  arXiv  Detail & Related papers  (2024-10-02T13:37:54Z)
• SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
  Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM. We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
  arXiv  Detail & Related papers  (2024-07-16T04:41:58Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users.<n>We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set.<n>We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks [22.66167973623777]
  Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning (ICL) capabilities. This paper investigates whether LLMs can generalize from labeled examples of predefined tasks to novel tasks. We show that cross-task prompting leads to a remarkable performance boost of 107% for LLaMA-2 7B, 18.6% for LLaMA-2 13B, and 3.2% for GPT 3.5 on average over zero-shot prompting.
  arXiv  Detail & Related papers  (2024-05-17T05:20:49Z)
• LLMaAA: Making Large Language Models as Active Annotators [32.57011151031332]
  We propose LLMaAA, which takes large language models as annotators and puts them into an active learning loop to determine what to annotate efficiently. We conduct experiments and analysis on two classic NLP tasks, named entity recognition and relation extraction. With LLMaAA, task-specific models trained from LLM-generated labels can outperform the teacher within only hundreds of annotated examples.
  arXiv  Detail & Related papers  (2023-10-30T14:54:15Z)
• TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models [52.734140807634624]
  Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. Existing continual learning benchmarks lack sufficient challenge for leading aligned LLMs. We introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
  arXiv  Detail & Related papers  (2023-10-10T16:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.