OpenAssistant Conversations -- Democratizing Large Language Model Alignment

• URL: http://arxiv.org/abs/2304.07327v2
• Date: Tue, 31 Oct 2023 11:38:07 GMT
• Title: OpenAssistant Conversations -- Democratizing Large Language Model Alignment
• Abstract summary: We release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers.
• Score: 37.617080996433586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 complete and fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Models trained on OpenAssistant Conversations show consistent improvements on standard benchmarks over respective base models. We release our code and data under a fully permissive licence.

Related papers

• OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models [55.63479003621053]
  We introduce OWLS, an open-access suite of multilingual speech recognition and translation models. We use OWLS to derive neural scaling laws, showing how final performance can be reliably predicted when scaling. We show how OWLS can be used to power new research directions by discovering emergent abilities in large-scale speech models.
  arXiv  Detail & Related papers  (2025-02-14T18:51:40Z)
• Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora [79.03392191805028]
  The BabyLM Challenge is a community effort to close the data-efficiency gap between human and computational language learners. Participants compete to optimize language model training on a fixed language data budget of 100 million words or less.
  arXiv  Detail & Related papers  (2024-12-06T16:06:08Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors. We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• Supporting Human Raters with the Detection of Harmful Content using Large Language Models [8.580258386804282]
  We demonstrate that large language models (LLMs) can achieve 90% accuracy when compared to human verdicts. We propose five design patterns that integrate LLMs with human rating. We share how piloting our proposed techniques in a real-world review queue yielded a 41.5% improvement in optimizing available human rater capacity.
  arXiv  Detail & Related papers  (2024-06-18T17:12:50Z)
• Data-Efficient Alignment of Large Language Models with Human Feedback Through Natural Language [31.0723480021355]
  We investigate data efficiency of modeling human feedback that is in natural language. We fine-tune an open-source LLM, e.g., Falcon-40B-Instruct, on a relatively small amount of human feedback in natural language. We show that this model is able to improve the quality of responses from even some of the strongest LLMs.
  arXiv  Detail & Related papers  (2023-11-24T15:20:36Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Enhancing Chat Language Models by Scaling High-quality Instructional Conversations [91.98516412612739]
  We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat. Our objective is to capture the breadth of interactions that a human might have with an AI assistant. We fine-tune a LLaMA model to create a powerful conversational model, UltraLLaMA.
  arXiv  Detail & Related papers  (2023-05-23T16:49:14Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing [73.16475763422446]
  We propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) It incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness.
  arXiv  Detail & Related papers  (2021-03-21T17:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.