ExpertPrompting: Instructing Large Language Models to be Distinguished Experts

• URL: http://arxiv.org/abs/2305.14688v1
• Date: Wed, 24 May 2023 03:51:31 GMT
• Title: ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
• Authors: Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, Zhendong Mao
• Abstract summary: ExpertPrompting elicits the potential of large language models to answer as distinguished experts. We produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA.
• Score: 93.58012324415762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts. In this paper, we propose ExpertPrompting to elicit the potential of LLMs to answer as distinguished experts. We first utilize In-Context Learning to automatically synthesize detailed and customized descriptions of the expert identity for each specific instruction, and then ask LLMs to provide answer conditioned on such agent background. Based on this augmented prompting strategy, we produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA. We employ GPT4-based evaluation to show that 1) the expert data is of significantly higher quality than vanilla answers, and 2) ExpertLLaMA outperforms existing open-source opponents and achieves 96\% of the original ChatGPT's capability. All data and the ExpertLLaMA model will be made publicly available at \url{https://github.com/OFA-Sys/ExpertLLaMA}.

Related papers

• Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM [1.3089936156875277]
  We introduce a speech-conditioned Large Language Model (LLM) integrated with a Mixture of Experts (MoE) based connector. We propose an Insertion and Deletion of Interruption Token (IDIT) mechanism for better transfer text generation ability of LLM to speech recognition task. We also present a connecter with MoE architecture that manages multiple languages efficiently.
  arXiv  Detail & Related papers  (2024-09-24T09:20:22Z)
• FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability [70.84333325049123]
  FoFo is a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats. This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats.
  arXiv  Detail & Related papers  (2024-02-28T19:23:27Z)
• Prompt-Time Symbolic Knowledge Capture with Large Language Models [0.0]
  Augmenting large language models (LLMs) with user-specific knowledge is crucial for real-world applications, such as personal AI assistants. This paper investigates utilizing the existing LLM capabilities to enable prompt-driven knowledge capture.
  arXiv  Detail & Related papers  (2024-02-01T08:15:28Z)
• PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization [60.00631098364391]
  PromptAgent is an optimization method that crafts expert-level prompts equivalent in quality to those handcrafted by experts. Inspired by human-like trial-and-error exploration, PromptAgent induces precise expert-level insights and in-depth instructions. We apply PromptAgent to 12 tasks spanning three practical domains.
  arXiv  Detail & Related papers  (2023-10-25T07:47:01Z)
• Tuna: Instruction Tuning using Feedback from Large Language Models [74.04950416204551]
  We propose finetuning an instruction-tuned large language model using our novel textitprobabilistic ranking and textitcontextual ranking approaches. Probabilistic ranking enables the instruction-tuned model to inherit the relative rankings of high-quality and low-quality responses from the teacher LLM. On the other hand, learning with contextual ranking allows the model to refine its own response distribution using the contextual understanding ability of stronger LLMs.
  arXiv  Detail & Related papers  (2023-10-20T09:55:06Z)
• Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models [53.09723678623779]
  We propose TAGREAL to automatically generate quality query prompts and retrieve support information from large text corpora. The results show that TAGREAL achieves state-of-the-art performance on two benchmark datasets. We find that TAGREAL has superb performance even with limited training data, outperforming existing embedding-based, graph-based, and PLM-based methods.
  arXiv  Detail & Related papers  (2023-05-24T22:09:35Z)
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators [98.11286353828525]
  GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. We propose AnnoLLM, which adopts a two-step approach, explain-then-annotate. We build the first conversation-based information retrieval dataset employing AnnoLLM.
  arXiv  Detail & Related papers  (2023-03-29T17:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.