Formalising lexical and syntactic diversity for data sampling in French

• URL: http://arxiv.org/abs/2501.08003v1
• Date: Tue, 14 Jan 2025 10:47:33 GMT
• Title: Formalising lexical and syntactic diversity for data sampling in French
• Abstract summary: We present a significantly increasing diversity relative to random sampling.<n>We explore whether different kinds of diversity -- lexical and syntactic -- correlate with the purpose of sampling for expensive syntactic diversity.<n>This shows that an arbitrarily chosen measure may fall short of capturing diversity-related properties of datasets.
• Score: 0.7947120571256026
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Diversity is an important property of datasets and sampling data for diversity is useful in dataset creation. Finding the optimally diverse sample is expensive, we therefore present a heuristic significantly increasing diversity relative to random sampling. We also explore whether different kinds of diversity -- lexical and syntactic -- correlate, with the purpose of sampling for expensive syntactic diversity through inexpensive lexical diversity. We find that correlations fluctuate with different datasets and versions of diversity measures. This shows that an arbitrarily chosen measure may fall short of capturing diversity-related properties of datasets.

Related papers

• emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity [4.597270663382362]
  We provide a comprehensive embedding-based diversity measurement tool spanning a broad range of measures.<n>We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets.
  arXiv  Detail & Related papers  (2026-07-22T07:33:55Z)
• Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error [19.594704501292778]
  We compare the diversity of samples generated by state-of-the-art models with that of test samples drawn from the target data distribution.<n>We find that test data consistently attains substantially higher Vendi and RKE diversity scores than the generated samples.
  arXiv  Detail & Related papers  (2026-02-16T12:15:34Z)
• Evaluating the Diversity and Quality of LLM Generated Content [72.84945252821908]
  We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds. Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models. These findings have important implications for applications that require diverse yet high-quality outputs.
  arXiv  Detail & Related papers  (2025-04-16T23:02:23Z)
• Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric [48.81957145701228]
  We propose a new diversity metric based on sample-level "novelty" We show that NovelSum accurately captures diversity variations and achieves a 0.97 correlation with instruction-tuned model performance.
  arXiv  Detail & Related papers  (2025-02-24T14:20:22Z)
• Diversity-oriented Data Augmentation with Large Language Models [9.548912625579947]
  We propose a textbfunderlineDi-textbfunderlineoriented data textbfunderlineAugmentation framework (textbfDoAug)<n>Specifically, we utilize a diversity-oriented fine-tuning approach to train an LLM as a diverse paraphraser, which is capable of augmenting textual datasets by generating diversified paraphrases.<n>The results show that our fine-tuned LLM augmenter improves diversity while preserving label consistency, thereby enhancing the robustness and performance of downstream tasks.
  arXiv  Detail & Related papers  (2025-02-17T11:00:40Z)
• Measuring Diversity in Synthetic Datasets [59.511610956067415]
  Large language models (LLMs) are widely adopted to generate synthetic datasets for various natural language processing (NLP) tasks. accurately measuring the diversity of these synthetic datasets-an aspect crucial for robust model performance-remains a significant challenge. We introduce DCScore, a novel method for measuring synthetic dataset diversity from a classification perspective.
  arXiv  Detail & Related papers  (2025-02-12T15:46:34Z)
• Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data [54.3895971080712]
  Fine-tuning large language models (LLMs) using diverse datasets is crucial for enhancing their overall performance across various domains.<n>We propose a new method that gives the LLM a dual identity: an output model to cognitively probe and select data based on diversity reward, as well as an input model to be tuned with the selected data.
  arXiv  Detail & Related papers  (2025-02-05T17:21:01Z)
• Exploring and Controlling Diversity in LLM-Agent Conversation [17.38671584773247]
  We propose Adaptive Prompt Pruning (APP), a novel method that dynamically adjusts the content of the utterance generation prompt to control diversity.<n>APP effectively controls the output diversity across models and datasets, with pruning more information leading to more diverse output.<n>APP is compatible with established techniques like temperature sampling and top-p sampling, providing a versatile tool for diversity management.
  arXiv  Detail & Related papers  (2024-12-30T17:25:58Z)
• Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL [51.48239006107272]
  In this paper, we discuss how to measure and improve the diversity of the demonstrations for text-to-diversity research. We propose fusing iteratively for demonstrations (Fused) to build a high-diversity demonstration pool. Our method achieves an average improvement of 3.2% and 5.0% with and without human labeling on several mainstream datasets.
  arXiv  Detail & Related papers  (2024-02-16T13:13:18Z)
• Diversity Measurement and Subset Selection for Instruction Tuning Datasets [40.930387018872786]
  We use determinantal point processes to capture the diversity and quality of instruction tuning datasets for subset selection. We propose to measure dataset diversity with log determinant distance that is the distance between the dataset of interest and a maximally diverse reference dataset.
  arXiv  Detail & Related papers  (2024-02-04T02:09:43Z)
• Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data [11.46958089612243]
  Current trends in pre-training Large Language Models (LLMs) primarily focus on the scaling of model and dataset size. While the quality of pre-training data is considered an important factor for training powerful LLMs, it remains a nebulous concept that has not been rigorously characterized. We propose a formalization of one key aspect of data quality -- measuring the variability of natural language data -- specifically via a measure we call the diversity coefficient.
  arXiv  Detail & Related papers  (2023-06-24T02:25:56Z)
• A Unified Theory of Diversity in Ensemble Learning [4.773356856466191]
  We present a theory of ensemble diversity, explaining the nature of diversity for a wide range of supervised learning scenarios. This challenge has been referred to as the holy grail of ensemble learning, an open research issue for over 30 years.
  arXiv  Detail & Related papers  (2023-01-10T13:51:07Z)
• Exploiting Diversity of Unlabeled Data for Label-Efficient Semi-Supervised Active Learning [57.436224561482966]
  Active learning is a research area that addresses the issues of expensive labeling by selecting the most important samples for labeling. We introduce a new diversity-based initial dataset selection algorithm to select the most informative set of samples for initial labeling in the active learning setting. Also, we propose a novel active learning query strategy, which uses diversity-based sampling on consistency-based embeddings.
  arXiv  Detail & Related papers  (2022-07-25T16:11:55Z)
• Exploring Diversity in Back Translation for Low-Resource Machine Translation [85.03257601325183]
  Back translation is one of the most widely used methods for improving the performance of neural machine translation systems. Recent research has sought to enhance the effectiveness of this method by increasing the 'diversity' of the generated translations. This work puts forward a more nuanced framework for understanding diversity in training data, splitting it into lexical diversity and syntactic diversity.
  arXiv  Detail & Related papers  (2022-06-01T15:21:16Z)
• Expressivity of Parameterized and Data-driven Representations in Quality Diversity Search [111.06379262544911]
  We compare the output diversity of a quality diversity evolutionary search performed in two different search spaces. A learned model is better at interpolating between known data points than at extrapolating or expanding towards unseen examples.
  arXiv  Detail & Related papers  (2021-05-10T10:27:43Z)
• Relationship-aware Multivariate Sampling Strategy for Scientific Simulation Data [4.2855912967712815]
  In this work, we propose a multivariate sampling strategy which preserves the original variable relationships. Our proposed strategy utilizes principal component analysis to capture the variance of multivariate data and can be built on top of any existing state-of-the-art sampling algorithms for single variables.
  arXiv  Detail & Related papers  (2020-08-31T00:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.