Small But Funny: A Feedback-Driven Approach to Humor Distillation
- URL: http://arxiv.org/abs/2402.18113v1
- Date: Wed, 28 Feb 2024 07:02:38 GMT
- Title: Small But Funny: A Feedback-Driven Approach to Humor Distillation
- Authors: Sahithya Ravi, Patrick Huber, Akshat Shrivastava, Aditya Sagar, Ahmed
Aly, Vered Shwartz, Arash Einolghozati
- Abstract summary: We study the effect of assigning a dual role to the Large Language Models (LLMs) - as a "teacher" generating data, and a "critic" evaluating the student's performance.
Our experiments on humor generation reveal that the incorporation of feedback significantly narrows the performance gap between SLMs and their larger counterparts.
- Score: 19.498647865012426
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The emergence of Large Language Models (LLMs) has brought to light promising
language generation capabilities, particularly in performing tasks like complex
reasoning and creative writing. Consequently, distillation through imitation of
teacher responses has emerged as a popular technique to transfer knowledge from
LLMs to more accessible, Small Language Models (SLMs). While this works well
for simpler tasks, there is a substantial performance gap on tasks requiring
intricate language comprehension and creativity, such as humor generation. We
hypothesize that this gap may stem from the fact that creative tasks might be
hard to learn by imitation alone and explore whether an approach, involving
supplementary guidance from the teacher, could yield higher performance. To
address this, we study the effect of assigning a dual role to the LLM - as a
"teacher" generating data, as well as a "critic" evaluating the student's
performance. Our experiments on humor generation reveal that the incorporation
of feedback significantly narrows the performance gap between SLMs and their
larger counterparts compared to merely relying on imitation. As a result, our
research highlights the potential of using feedback as an additional dimension
to data when transferring complex language abilities via distillation.
Related papers
- Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation [15.718288693929019]
Large Language Models (LLM) achieve state-of-the-art performance on many NLP tasks.
We study whether LLMs can be used as substitutes for human annotators.
We find that LLMs outperform current automatic measures for system-level evaluation but still struggle to provide satisfactory explanations.
arXiv Detail & Related papers (2024-05-22T15:56:52Z) - I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses [23.053791342294268]
fine-tuning a large language model (LLM) with responses generated by a LLM often yields better results than using responses generated by humans.
Training with LLM-generated responses not only enhances performance but also helps maintain the model's capabilities in other tasks after fine-tuning on a specific task.
arXiv Detail & Related papers (2024-02-17T05:05:31Z) - Unmemorization in Large Language Models via Self-Distillation and
Deliberate Imagination [58.36408867180233]
Large Language Models (LLMs) struggle with crucial issues of privacy violation and unwanted exposure of sensitive data.
We introduce a novel approach termed deliberate imagination in the context of LLM unlearning.
Our results demonstrate the usefulness of this approach across different models and sizes, and also with parameter-efficient fine-tuning.
arXiv Detail & Related papers (2024-02-15T16:21:14Z) - TinyLLM: Learning a Small Student from Multiple Large Language Models [23.736611338497244]
TinyLLM is a new knowledge distillation paradigm to learn a small student LLM from multiple large teacher LLMs.
We introduce an in-context example generator and a teacher-forcing Chain-of-Thought strategy to ensure that the rationales are accurate and grounded in contextually appropriate scenarios.
arXiv Detail & Related papers (2024-02-07T06:48:24Z) - AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations [52.43593893122206]
AlignedCoT is an in-context learning technique for invoking Large Language Models.
It achieves consistent and correct step-wise prompts in zero-shot scenarios.
We conduct experiments on mathematical reasoning and commonsense reasoning.
arXiv Detail & Related papers (2023-11-22T17:24:21Z) - Democratizing Reasoning Ability: Tailored Learning from Large Language
Model [97.4921006089966]
We propose a tailored learning approach to distill such reasoning ability to smaller LMs.
We exploit the potential of LLM as a reasoning teacher by building an interactive multi-round learning paradigm.
To exploit the reasoning potential of the smaller LM, we propose self-reflection learning to motivate the student to learn from self-made mistakes.
arXiv Detail & Related papers (2023-10-20T07:50:10Z) - Self-Convinced Prompting: Few-Shot Question Answering with Repeated
Introspection [13.608076739368949]
We introduce a novel framework that harnesses the potential of large-scale pre-trained language models.
Our framework processes the output of a typical few-shot chain-of-thought prompt, assesses the correctness of the response, scrutinizes the answer, and ultimately produces a new solution.
arXiv Detail & Related papers (2023-10-08T06:36:26Z) - Are Large Language Models Really Robust to Word-Level Perturbations? [68.60618778027694]
We propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools.
Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions.
Our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage.
arXiv Detail & Related papers (2023-09-20T09:23:46Z) - Improving Factuality and Reasoning in Language Models through Multiagent
Debate [95.10641301155232]
We present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer.
Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks.
Our approach may be directly applied to existing black-box models and uses identical procedure and prompts for all tasks we investigate.
arXiv Detail & Related papers (2023-05-23T17:55:11Z) - Computational Language Acquisition with Theory of Mind [84.2267302901888]
We build language-learning agents equipped with Theory of Mind (ToM) and measure its effects on the learning process.
We find that training speakers with a highly weighted ToM listener component leads to performance gains in our image referential game setting.
arXiv Detail & Related papers (2023-03-02T18:59:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.