Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation

• URL: http://arxiv.org/abs/2209.00455v1
• Date: Wed, 31 Aug 2022 06:59:36 GMT
• Title: Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation
• Authors: Sirui Wang, Kaiwen Wei, Hongzhi Zhang, Yuntao Li and Wei Wu
• Abstract summary: Demonstration learning aims to guide the prompt prediction via providing answered demonstrations in the few shot settings. Existing work onlycorporas the answered examples as demonstrations to the prompt template without any additional operation. We introduce Imitation DEMOnstration Learning (Imitation-Demo) to strengthen demonstration learning via explicitly imitating human review behaviour.
• Score: 9.851250429233634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Demonstration learning aims to guide the prompt prediction via providing answered demonstrations in the few shot settings. Despite achieving promising results, existing work only concatenates the answered examples as demonstrations to the prompt template (including the raw context) without any additional operation, neglecting the prompt-demonstration dependencies. Besides, prior research found that randomly replacing the labels of demonstrations marginally hurts performance, illustrating that the model could not properly learn the knowledge brought by the demonstrations. Inspired by the human learning process, in this paper, we introduce Imitation DEMOnstration Learning (Imitation-Demo) to strengthen demonstration learning via explicitly imitating human review behaviour, which includes: (1) contrastive learning mechanism to concentrate on the similar demonstrations. (2) demonstration-label re-prediction method to consolidate known knowledge. Experiment results show that our proposed method achieves state-of-the-art performance on 11 out of 14 classification corpora. Further studies also prove that Imitation-Demo strengthen the association between prompt and demonstrations, which could provide the basis for exploring how demonstration learning works.

Related papers

• Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions [8.869100154323643]
  We propose a novel prompt engineering workflow built around a novel object called the "demonstration notebook" This notebook helps identify the most suitable in-context learning example for a question by gathering and reusing information from the LLM's past interactions. Our experiments show that this approach outperforms all existing methods for automatic demonstration construction and selection.
  arXiv  Detail & Related papers  (2024-06-16T10:02:20Z)
• Imitation Learning from Purified Demonstration [55.23663861003027]
  We propose to purify the potential perturbations in imperfect demonstrations and conduct imitation learning from purified demonstrations. We provide theoretical evidence supporting our approach, demonstrating that total variance distance between the purified and optimal demonstration distributions can be upper-bounded.
  arXiv  Detail & Related papers  (2023-10-11T02:36:52Z)
• Scaling In-Context Demonstrations with Structured Attention [75.41845145597875]
  We propose a better architectural design for in-context learning. Structured Attention for In-Context Learning replaces the full-attention by a structured attention mechanism. We show that SAICL achieves comparable or better performance than full attention while obtaining up to 3.4x inference speed-up.
  arXiv  Detail & Related papers  (2023-07-05T23:26:01Z)
• Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations [60.241144377865716]
  We consider the imitation of sub-optimal demonstrations, with both a small clean demonstration set and a large noisy set. We propose method by evaluating and imitating at the sub-demonstration level, encoding action primitives of varying quality into different skills.
  arXiv  Detail & Related papers  (2023-06-13T17:24:37Z)
• Boosting Reinforcement Learning and Planning with Demonstrations: A Survey [25.847796336059343]
  We discuss the advantages of using demonstrations in sequential decision making. We exemplify a practical pipeline for generating and utilizing demonstrations in the recently proposed ManiSkill robot learning benchmark.
  arXiv  Detail & Related papers  (2023-03-23T17:53:44Z)
• Out-of-Dynamics Imitation Learning from Multimodal Demonstrations [68.46458026983409]
  We study out-of-dynamics imitation learning (OOD-IL), which relaxes the assumption to that the demonstrator and the imitator have the same state spaces. OOD-IL enables imitation learning to utilize demonstrations from a wide range of demonstrators but introduces a new challenge. We develop a better transferability measurement to tackle this newly-emerged challenge.
  arXiv  Detail & Related papers  (2022-11-13T07:45:06Z)
• Robustness of Demonstration-based Learning Under Limited Data Scenario [54.912936555876826]
  Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario. Why such demonstrations are beneficial for the learning process remains unclear since there is no explicit alignment between the demonstrations and the predictions. In this paper, we design pathological demonstrations by gradually removing intuitively useful information from the standard ones to take a deep dive of the robustness of demonstration-based sequence labeling.
  arXiv  Detail & Related papers  (2022-10-19T16:15:04Z)
• Extraneousness-Aware Imitation Learning [25.60384350984274]
  Extraneousness-Aware Learning (EIL) learns visuomotor policies from third-person demonstrations with extraneous subsequences. EIL learns action-conditioned observation embeddings in a self-supervised manner and retrieves task-relevant observations across visual demonstrations. Experimental results show that EIL outperforms strong baselines and achieves comparable policies to those trained with perfect demonstration.
  arXiv  Detail & Related papers  (2022-10-04T04:42:26Z)
• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? [112.72413411257662]
  Large language models (LMs) are able to in-context learn by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. We show that ground truth demonstrations are in fact not required -- randomly replacing labels in the demonstrations barely hurts performance. We find that other aspects of the demonstrations are the key drivers of end task performance.
  arXiv  Detail & Related papers  (2022-02-25T17:25:19Z)
• Learning Feasibility to Imitate Demonstrators with Different Dynamics [23.239058855103067]
  The goal of learning from demonstrations is to learn a policy for an agent (imitator) by mimicking the behavior in the demonstrations. We learn a feasibility metric that captures the likelihood of a demonstration being feasible by the imitator. Our experiments on four simulated environments and on a real robot show that the policy learned with our approach achieves a higher expected return than prior works.
  arXiv  Detail & Related papers  (2021-10-28T14:15:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.