Evaluating the Diversity of AI-Generated Content with Diversity Profiles

• URL: http://arxiv.org/abs/2608.17731v1
• Date: Tue, 18 Aug 2026 12:57:03 GMT
• Title: Evaluating the Diversity of AI-Generated Content with Diversity Profiles
• Abstract summary: We argue that diversity evaluation for AI-generated content is intrinsically under-specified when reduced to a single number.<n>We propose diversity profiles that evaluate a parameterized diversity family across a range of thresholds, scales, exponents, or orders.<n>Overall, diversity profiles provide a more transparent and resolution-aware framework for comparing the diversity of AI-generated content.
• Score: 37.17952750848635
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diversity is a fundamental criterion for evaluating generative artificial intelligence (AI) systems, yet its measurement remains inherently ambiguous. Existing approaches typically represent generated samples in an embedding space, compute pairwise distances or similarities, and aggregate them into a single scalar score. Such scalar summaries are convenient, but they often encode different inductive biases and may yield contradictory rankings of the same sample sets. In this paper, we argue that diversity evaluation for AI-generated content is intrinsically under-specified when reduced to a single number. We first review representative diversity metrics, and then diagnose their limitations from two complementary perspectives: an axiomatic analysis showing that no representative scalar metric satisfies all desirable properties simultaneously, and an empirical analysis showing that high-dimensional representation spaces can induce concentrated, modality-dependent distance distributions. To address these issues, we propose diversity profiles: curve-valued, condition-aware summaries that evaluate a parameterized diversity family across a range of thresholds, scales, exponents, or orders under a specified representation and distance or kernel function. Diversity profiles reveal whether a comparison is robust across resolutions or instead depends on an arbitrary parameter choice. We instantiate profiles for several representative metric families and demonstrate their practical use in generative AI evaluation. Overall, diversity profiles provide a more transparent and resolution-aware framework for comparing the diversity of AI-generated content.

Related papers

• Learning Unified Distance Metric for Heterogeneous Attribute Data Clustering [60.05209293008078]
  Heterogeneous Attribute Reconstruction and Representation (HARR) learning paradigm for cluster analysis.<n>HarR is parameter-free, convergence-guaranteed, and can more effectively self-adapt to different sought number of clusters $k$.
  arXiv  Detail & Related papers  (2026-03-03T08:13:16Z)
• SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score [22.364171814102487]
  Diffusion models have demonstrated remarkable success in high-fidelity image synthesis and prompt-guided generative modeling.<n>We propose the Scalable Prompt-Aware R'eny Kernel Entropy Diversity Guidance (SPARKE) method for prompt-aware diversity guidance.<n>We numerically test the SPARKE method on several text-to-image diffusion models, demonstrating that the proposed method improves the prompt-aware diversity of the generated data without incurring significant computational costs.
  arXiv  Detail & Related papers  (2025-06-11T20:53:45Z)
• Evaluating the Diversity and Quality of LLM Generated Content [72.84945252821908]
  We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds.<n>Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models.<n>These findings have important implications for applications that require diverse yet high-quality outputs.
  arXiv  Detail & Related papers  (2025-04-16T23:02:23Z)
• Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric [48.81957145701228]
  We propose NovelSum, a new diversity metric based on sample-level "novelty"<n> Experiments on both simulated and real-world data show that NovelSum accurately captures diversity variations and achieves a 0.97 correlation with instruction-tuned model performance.
  arXiv  Detail & Related papers  (2025-02-24T14:20:22Z)
• A Unifying Information-theoretic Perspective on Evaluating Generative Models [5.524685807042777]
  Several recent approaches utilize "precision" and "recall," borrowed from the classification domain, to individually quantify the output fidelity (realism) and output diversity (representation of the real data variation)<n>We unify a class of kth-nearest-neighbors (kNN)-based metrics under an information-theoretic lens using approaches from kNN density estimation.<n>We propose a tri-dimensional metric composed of Precision Cross-Entropy (PCE), Recall Cross-Entropy (RCE), and Recall Entropy (RE)
  arXiv  Detail & Related papers  (2024-12-18T21:17:02Z)
• Comparative Analysis of Indicators for Multiobjective Diversity Optimization [0.2144088660722956]
  We discuss different diversity indicators from the perspective of indicator-based evolutionary algorithms (IBEA) with multiple objectives. We examine theoretical, computational, and practical properties of these indicators, such as monotonicity in species. We present new theorems -- including a proof of the NP-hardness of the Riesz s-Energy Subset Selection Problem.
  arXiv  Detail & Related papers  (2024-10-24T16:40:36Z)
• Contributing Dimension Structure of Deep Feature for Coreset Selection [26.759457501199822]
  Coreset selection seeks to choose a subset of crucial training samples for efficient learning. Sample selection hinges on two main aspects: a sample's representation in enhancing performance and the role of sample diversity in averting overfitting. Existing methods typically measure both the representation and diversity of data based on similarity metrics.
  arXiv  Detail & Related papers  (2024-01-29T14:47:26Z)
• Interpretable Diversity Analysis: Visualizing Feature Representations In Low-Cost Ensembles [0.0]
  This paper introduces several interpretability methods that can be used to qualitatively analyze diversity. We demonstrate these techniques by comparing the diversity of feature representations between child networks using two low-cost ensemble algorithms.
  arXiv  Detail & Related papers  (2023-02-12T00:32:03Z)
• Deep Diversity-Enhanced Feature Representation of Hyperspectral Images [87.47202258194719]
  We rectify 3D convolution by modifying its topology to enhance the rank upper-bound. We also propose a novel diversity-aware regularization (DA-Reg) term that acts on the feature maps to maximize independence among elements. To demonstrate the superiority of the proposed Re$3$-ConvSet and DA-Reg, we apply them to various HS image processing and analysis tasks.
  arXiv  Detail & Related papers  (2023-01-15T16:19:18Z)
• Expressivity of Parameterized and Data-driven Representations in Quality Diversity Search [111.06379262544911]
  We compare the output diversity of a quality diversity evolutionary search performed in two different search spaces. A learned model is better at interpolating between known data points than at extrapolating or expanding towards unseen examples.
  arXiv  Detail & Related papers  (2021-05-10T10:27:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.