emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity

• URL: http://arxiv.org/abs/2607.19848v1
• Date: Wed, 22 Jul 2026 07:33:55 GMT
• Title: emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity
• Abstract summary: We provide a comprehensive embedding-based diversity measurement tool spanning a broad range of measures.<n>We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets.
• Score: 4.597270663382362
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools for measuring the lexical diversity of texts, researchers lack standardized tools for quantifying diversity based on embeddings. Embedding-based diversity measures are highly flexible: They work with any embedding model and any data that can be embedded, and are thus applicable to many notions of diversity. With emb-diversity, we provide a comprehensive embedding-based diversity measurement tool, spanning a broad range of measures. We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets. https://github.com/nlpsoc/emb-diversity/

Related papers

• Expanding Horizons of Level Diversity via Multi-objective Evolutionary Learning [10.755666953578336]
  This paper aims to expand horizons of level diversity by considering multi-dimensional diversity when training generative models.<n>We formulate the model training as a multi-objective learning problem, where each diversity metric is treated as a distinct objective.<n>A multi-objective evolutionary learning framework that optimises multiple diversity metrics simultaneously throughout the model training process is proposed.
  arXiv  Detail & Related papers  (2025-09-29T06:43:33Z)
• Evaluating the Evaluation of Diversity in Commonsense Generation [28.654890118684957]
  We conduct a systematic meta-evaluation of diversity metrics for commonsense generation.<n>We find that form-based diversity metrics tend to consistently overestimate the diversity in sentence sets.<n>We show that content-based diversity evaluation metrics consistently outperform the form-based counterparts.
  arXiv  Detail & Related papers  (2025-05-31T11:18:26Z)
• Evaluating the Diversity and Quality of LLM Generated Content [72.84945252821908]
  We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds.<n>Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models.<n>These findings have important implications for applications that require diverse yet high-quality outputs.
  arXiv  Detail & Related papers  (2025-04-16T23:02:23Z)
• Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric [48.81957145701228]
  We propose NovelSum, a new diversity metric based on sample-level "novelty"<n> Experiments on both simulated and real-world data show that NovelSum accurately captures diversity variations and achieves a 0.97 correlation with instruction-tuned model performance.
  arXiv  Detail & Related papers  (2025-02-24T14:20:22Z)
• Formalising lexical and syntactic diversity for data sampling in French [0.7947120571256026]
  We present a significantly increasing diversity relative to random sampling.<n>We explore whether different kinds of diversity -- lexical and syntactic -- correlate with the purpose of sampling for expensive syntactic diversity.<n>This shows that an arbitrarily chosen measure may fall short of capturing diversity-related properties of datasets.
  arXiv  Detail & Related papers  (2025-01-14T10:47:33Z)
• Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL [51.48239006107272]
  In this paper, we discuss how to measure and improve the diversity of the demonstrations for text-to-diversity research. We propose fusing iteratively for demonstrations (Fused) to build a high-diversity demonstration pool. Our method achieves an average improvement of 3.2% and 5.0% with and without human labeling on several mainstream datasets.
  arXiv  Detail & Related papers  (2024-02-16T13:13:18Z)
• Exploring Diversity in Back Translation for Low-Resource Machine Translation [85.03257601325183]
  Back translation is one of the most widely used methods for improving the performance of neural machine translation systems. Recent research has sought to enhance the effectiveness of this method by increasing the 'diversity' of the generated translations. This work puts forward a more nuanced framework for understanding diversity in training data, splitting it into lexical diversity and syntactic diversity.
  arXiv  Detail & Related papers  (2022-06-01T15:21:16Z)
• Random Network Distillation as a Diversity Metric for Both Image and Text Generation [62.13444904851029]
  We develop a new diversity metric that can be applied to data, both synthetic and natural, of any type. We validate and deploy this metric on both images and text.
  arXiv  Detail & Related papers  (2020-10-13T22:03:52Z)
• Evaluating the Evaluation of Diversity in Natural Language Generation [43.05127848086264]
  We propose a framework for evaluating diversity metrics in natural language generation systems. Our framework can advance the understanding of different diversity metrics, an essential step on the road towards better NLG systems.
  arXiv  Detail & Related papers  (2020-04-06T20:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.