Evaluating the Diversity and Quality of LLM Generated Content

• URL: http://arxiv.org/abs/2504.12522v1
• Date: Wed, 16 Apr 2025 23:02:23 GMT
• Title: Evaluating the Diversity and Quality of LLM Generated Content
• Authors: Alexander Shypula, Shuo Li, Botong Zhang, Vishakh Padmakumar, Kayo Yin, Osbert Bastani,
• Abstract summary: We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds.<n>Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models.<n>These findings have important implications for applications that require diverse yet high-quality outputs.
• Score: 72.84945252821908
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work suggests that preference-tuning techniques--including Reinforcement Learning from Human Preferences (RLHF) methods like PPO and GRPO, as well as alternatives like DPO--reduce diversity, creating a dilemma given that such models are widely deployed in applications requiring diverse outputs. To address this, we introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds--which better reflects the practical utility of large language models (LLMs). Using open-ended tasks that require no human intervention, we find counterintuitive results: although preference-tuned models--especially those trained via RL--exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models, not from increasing diversity among high-quality outputs, but from generating more high-quality outputs overall. We discover that preference tuning reduces syntactic diversity while preserving semantic diversity--revealing a distinction between diversity in form and diversity in content that traditional metrics often overlook. Our analysis further shows that smaller models are consistently more parameter-efficient at generating unique content within a fixed sampling budget, offering insights into the relationship between model scaling and diversity. These findings have important implications for applications that require diverse yet high-quality outputs, from creative assistance to synthetic data generation.

Related papers

• Modifying Large Language Model Post-Training for Diverse Creative Writing [12.872333448726595]
  In creative writing generation, we investigate post-training approaches to promote both output diversity and quality.<n>Our core idea is to include deviation in the training objective to facilitate learning from rare high-quality instances.<n>Our best model with 8B parameters could achieve on-par diversity as a human-created dataset while having output quality similar to the best instruction-tuned models.
  arXiv  Detail & Related papers  (2025-03-21T13:21:45Z)
• Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric [48.81957145701228]
  We propose a new diversity metric based on sample-level "novelty" We show that NovelSum accurately captures diversity variations and achieves a 0.97 correlation with instruction-tuned model performance.
  arXiv  Detail & Related papers  (2025-02-24T14:20:22Z)
• Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting [9.116108409344177]
  The source-free cross-domain few-shot learning task aims to transfer pretrained models to target domains utilizing minimal samples.<n>We propose the SeGD-VPT framework, which is divided into two phases.<n>The first step aims to increase feature diversity by adding diversity prompts to each support sample, thereby generating varying input and enhancing sample diversity.
  arXiv  Detail & Related papers  (2024-12-01T11:00:38Z)
• Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning [28.654890118684957]
  Generative Commonsense Reasoning (GCR) requires a model to reason about a situation using commonsense knowledge. The diversity of the generation is equally important because it reflects the model's ability to use a range of commonsense knowledge facts. We propose a simple method that diversifies the LLM generations, while preserving their quality.
  arXiv  Detail & Related papers  (2024-04-25T17:52:39Z)
• Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization [13.436983663467938]
  This paper introduces Quality Diversity through Human Feedback (QDHF), a novel approach that progressively infers diversity metrics from human judgments of similarity among solutions. Empirical studies show that QDHF significantly outperforms state-of-the-art methods in automatic diversity discovery. In open-ended generative tasks, QDHF substantially enhances the diversity of text-to-image generation from a diffusion model.
  arXiv  Detail & Related papers  (2023-10-18T16:46:16Z)
• Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks [92.32670915472099]
  We propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs) We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.
  arXiv  Detail & Related papers  (2023-10-03T17:37:52Z)
• Source-free Domain Adaptation Requires Penalized Diversity [60.04618512479438]
  Source-free domain adaptation (SFDA) was introduced to address knowledge transfer between different domains in the absence of source data. In unsupervised SFDA, the diversity is limited to learning a single hypothesis on the source or learning multiple hypotheses with a shared feature extractor. We propose a novel unsupervised SFDA algorithm that promotes representational diversity through the use of separate feature extractors.
  arXiv  Detail & Related papers  (2023-04-06T00:20:19Z)
• Exploring Diversity in Back Translation for Low-Resource Machine Translation [85.03257601325183]
  Back translation is one of the most widely used methods for improving the performance of neural machine translation systems. Recent research has sought to enhance the effectiveness of this method by increasing the 'diversity' of the generated translations. This work puts forward a more nuanced framework for understanding diversity in training data, splitting it into lexical diversity and syntactic diversity.
  arXiv  Detail & Related papers  (2022-06-01T15:21:16Z)
• Informed Sampling for Diversity in Concept-to-Text NLG [8.883733362171034]
  We propose an Imitation Learning approach to explore the level of diversity that a language generation model can reliably produce. Specifically, we augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output.
  arXiv  Detail & Related papers  (2020-04-29T17:43:24Z)
• Towards Multimodal Response Generation with Exemplar Augmentation and Curriculum Optimization [73.45742420178196]
  We propose a novel multimodal response generation framework with exemplar augmentation and curriculum optimization. Our model achieves significant improvements compared to strong baselines in terms of diversity and relevance.
  arXiv  Detail & Related papers  (2020-04-26T16:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.