TPD: Enhancing Student Language Model Reasoning via Principle Discovery
and Guidance
- URL: http://arxiv.org/abs/2401.13849v1
- Date: Wed, 24 Jan 2024 23:11:33 GMT
- Title: TPD: Enhancing Student Language Model Reasoning via Principle Discovery
and Guidance
- Authors: Haorui Wang (1), Rongzhi Zhang (1), Yinghao Li (1), Lingkai Kong (1),
Yuchen Zhuang (1), Xiusi Chen (2), Chao Zhang (1) ((1) College of Computing,
Georgia Institute of Technology, (2) Department of Computer Science,
University of California, Los Angeles)
- Abstract summary: We introduce a principle-based teacher-student framework called Teaching via Principle Discovery'' (TPD)
Inspired by human learning mechanisms, TPD mimics the interaction between a teacher and a student using a principle-based approach.
TPD significantly improves the student model's performance, achieving $6.2%$ improvement on average.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large Language Models (LLMs) have recently showcased remarkable reasoning
abilities. However, larger models often surpass their smaller counterparts in
reasoning tasks, posing the challenge of effectively transferring these
capabilities from larger models. Existing approaches heavily rely on extensive
fine-tuning data or continuous interactions with a superior teacher LLM during
inference. We introduce a principle-based teacher-student framework called
``Teaching via Principle Discovery'' (TPD) to address these limitations.
Inspired by human learning mechanisms, TPD mimics the interaction between a
teacher and a student using a principle-based approach. The teacher LLM
generates problem-solving instructions and corrective principles based on the
student LLM's errors. These principles guide the refinement of instructions and
the selection of instructive examples from a validation set. This enables the
student model to learn from both the teacher's guidance and its own mistakes.
Once the student model begins making inferences, TPD requires no further
intervention from the teacher LLM or humans. Through extensive experiments
across eight reasoning tasks, we demonstrate the effectiveness of TPD. Compared
to standard chain-of-thought prompting, TPD significantly improves the student
model's performance, achieving $6.2\%$ improvement on average.
Related papers
- Retrieved In-Context Principles from Previous Mistakes [55.109234526031884]
In-context learning (ICL) has been instrumental in adapting Large Language Models (LLMs) to downstream tasks using correct input-output examples.
Recent advances have attempted to improve model performance through principles derived from mistakes.
We propose Retrieved In-Context Principles (RICP), a novel teacher-student framework.
arXiv Detail & Related papers (2024-07-08T07:32:26Z) - Aligning Teacher with Student Preferences for Tailored Training Data Generation [40.85451525264779]
We propose ARTE, dubbed Aligning TeacheR with StudenT PreferencEs, to generate tailored training examples for Knowledge Distillation.
Specifically, we elicit draft questions and rationales from the teacher model, then collect student preferences on these questions and rationales.
In the end, we repeat the first step with the aligned teacher model to elicit tailored training examples for the student model on the target task.
arXiv Detail & Related papers (2024-06-27T14:51:17Z) - PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs [47.35598271306371]
Large Language Models (LLMs) have exhibited impressive capabilities in various tasks, yet their vast parameter sizes restrict their applicability in resource-constrained settings.
Knowledge distillation (KD) offers a viable solution by transferring expertise from large teacher models to compact student models.
We present PLaD, a novel preference-based LLM distillation framework.
arXiv Detail & Related papers (2024-06-05T03:08:25Z) - Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models [39.82130327284791]
Large Language Models (LLMs) have demonstrated impressive zero-shot capabilities and versatility in NLP tasks.
They sometimes fail to maintain crucial invariances for specific tasks.
This paper addresses this inefficiency at inference time.
arXiv Detail & Related papers (2024-03-20T13:38:07Z) - YODA: Teacher-Student Progressive Learning for Language Models [82.0172215948963]
This paper introduces YODA, a teacher-student progressive learning framework.
It emulates the teacher-student education process to improve the efficacy of model fine-tuning.
Experiments show that training LLaMA2 with data from YODA improves SFT with significant performance gain.
arXiv Detail & Related papers (2024-01-28T14:32:15Z) - SALMON: Self-Alignment with Instructable Reward Models [80.83323636730341]
This paper presents a novel approach, namely SALMON, to align base language models with minimal human supervision.
We develop an AI assistant named Dromedary-2 with only 6 exemplars for in-context learning and 31 human-defined principles.
arXiv Detail & Related papers (2023-10-09T17:56:53Z) - DriveAdapter: Breaking the Coupling Barrier of Perception and Planning
in End-to-End Autonomous Driving [64.57963116462757]
State-of-the-art methods usually follow the Teacher-Student' paradigm.
Student model only has access to raw sensor data and conducts behavior cloning on the data collected by the teacher model.
We propose DriveAdapter, which employs adapters with the feature alignment objective function between the student (perception) and teacher (planning) modules.
arXiv Detail & Related papers (2023-08-01T09:21:53Z) - Distantly-Supervised Named Entity Recognition with Adaptive Teacher
Learning and Fine-grained Student Ensemble [56.705249154629264]
Self-training teacher-student frameworks are proposed to improve the robustness of NER models.
In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks.
Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
arXiv Detail & Related papers (2022-12-13T12:14:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.