LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and
Generative Fusion
- URL: http://arxiv.org/abs/2306.02561v3
- Date: Fri, 30 Jun 2023 21:39:54 GMT
- Title: LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and
Generative Fusion
- Authors: Dongfu Jiang, Xiang Ren, Bill Yuchen Lin
- Abstract summary: Our framework consists of two modules: PairRanker and GenFuser.
PairRanker employs a specialized pairwise comparison method to distinguish subtle differences between candidate outputs.
GenFuser aims to merge the top-ranked candidates, generating an improved output.
- Score: 33.73671362609599
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present LLM-Blender, an ensembling framework designed to attain
consistently superior performance by leveraging the diverse strengths of
multiple open-source large language models (LLMs). Our framework consists of
two modules: PairRanker and GenFuser, addressing the observation that optimal
LLMs for different examples can significantly vary. PairRanker employs a
specialized pairwise comparison method to distinguish subtle differences
between candidate outputs. It jointly encodes the input text and a pair of
candidates, using cross-attention encoders to determine the superior one. Our
results demonstrate that PairRanker exhibits the highest correlation with
ChatGPT-based ranking. Then, GenFuser aims to merge the top-ranked candidates,
generating an improved output by capitalizing on their strengths and mitigating
their weaknesses. To facilitate large-scale evaluation, we introduce a
benchmark dataset, MixInstruct, which is a mixture of multiple instruction
datasets featuring oracle pairwise comparisons. Our LLM-Blender significantly
outperform individual LLMs and baseline methods across various metrics,
establishing a substantial performance gap.
Related papers
- Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching [47.01589023992927]
We design a compound entity matching framework (ComEM) that leverages the composition of multiple strategies and large language models (LLMs)
ComEM benefits from the advantages of different sides and achieves improvements in both effectiveness and efficiency.
Experimental results on 8 ER datasets and 9 LLMs verify the superiority of incorporating record interactions through the selecting strategy.
arXiv Detail & Related papers (2024-05-27T07:05:27Z) - Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts [54.529880848937104]
We develop a unified MLLM with the MoE architecture, named Uni-MoE, that can handle a wide array of modalities.
Specifically, it features modality-specific encoders with connectors for a unified multimodal representation.
We evaluate the instruction-tuned Uni-MoE on a comprehensive set of multimodal datasets.
arXiv Detail & Related papers (2024-05-18T12:16:01Z) - Automated Multi-level Preference for MLLMs [41.72392895643214]
Current multimodal Large Language Models (MLLMs) suffer from hallucination''
One promising path is to utilize reinforcement learning from human feedback (RLHF), which steers MLLMs towards learning superior responses while avoiding inferior ones.
We rethink the common practice of using binary preferences (i.e., superior, inferior), and find that adopting multi-level preferences is better for two benefits.
arXiv Detail & Related papers (2024-05-18T03:49:37Z) - Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond [24.151927600694066]
Commit Message Generation (CMG) approaches aim to automatically generate commit messages based on given code diffs.
This paper conducts the first comprehensive experiment to investigate how far we have been in applying Large Language Models (LLMs) to generate high-quality commit messages.
arXiv Detail & Related papers (2024-04-23T08:24:43Z) - Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration [39.35476224845088]
Large language models (LLMs) exhibit complementary strengths in various tasks, motivating the research of LLM ensembling.
We propose a training-free ensemble framework DeePEn, fusing the informative probability distributions yielded by different LLMs at each decoding step.
arXiv Detail & Related papers (2024-04-19T08:52:22Z) - Make Large Language Model a Better Ranker [20.532118635672763]
This paper introduces the large language model framework with Aligned Listwise Ranking Objectives (ALRO)
ALRO is designed to bridge the gap between the capabilities of LLMs and nuanced requirements of ranking tasks.
Our evaluative studies reveal that ALRO outperforms both existing embedding-based recommendation methods and LLM-based recommendation baselines.
arXiv Detail & Related papers (2024-03-28T07:22:16Z) - Routing to the Expert: Efficient Reward-guided Ensemble of Large
Language Models [69.51130760097818]
We propose Zooter, a reward-guided routing method distilling rewards on training queries to train a routing function.
We evaluate Zooter on a comprehensive benchmark collection with 26 subsets on different domains and tasks.
arXiv Detail & Related papers (2023-11-15T04:40:43Z) - MLLM-DataEngine: An Iterative Refinement Approach for MLLM [62.30753425449056]
We propose a novel closed-loop system that bridges data generation, model training, and evaluation.
Within each loop, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results.
For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data.
For quality, we resort to GPT-4 to generate high-quality data with each given data type.
arXiv Detail & Related papers (2023-08-25T01:41:04Z) - LLMRec: Benchmarking Large Language Models on Recommendation Task [54.48899723591296]
The application of Large Language Models (LLMs) in the recommendation domain has not been thoroughly investigated.
We benchmark several popular off-the-shelf LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization.
The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation.
arXiv Detail & Related papers (2023-08-23T16:32:54Z) - From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets.
Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
arXiv Detail & Related papers (2023-08-23T09:45:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.