Revealing Trends in Datasets from the 2022 ACL and EMNLP Conferences

• URL: http://arxiv.org/abs/2404.08666v2
• Date: Mon, 15 Jul 2024 14:07:16 GMT
• Title: Revealing Trends in Datasets from the 2022 ACL and EMNLP Conferences
• Authors: Jesse Atuhurra, Hidetaka Kamigaito,
• Abstract summary: Transformers have given birth to pre-trained large language models (PLMs) The need to have quality datasets has prompted NLP researchers to continue creating new datasets to satisfy particular needs. This work aims to uncover the trends and insights mined within these datasets.
• Score: 16.66079305798581
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language processing (NLP) has grown significantly since the advent of the Transformer architecture. Transformers have given birth to pre-trained large language models (PLMs). There has been tremendous improvement in the performance of NLP systems across several tasks. NLP systems are on par or, in some cases, better than humans at accomplishing specific tasks. However, it remains the norm that \emph{better quality datasets at the time of pretraining enable PLMs to achieve better performance, regardless of the task.} The need to have quality datasets has prompted NLP researchers to continue creating new datasets to satisfy particular needs. For example, the two top NLP conferences, ACL and EMNLP, accepted ninety-two papers in 2022, introducing new datasets. This work aims to uncover the trends and insights mined within these datasets. Moreover, we provide valuable suggestions to researchers interested in curating datasets in the future.

Related papers

• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes [57.62036621319563]
  We introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. We demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators.
  arXiv  Detail & Related papers  (2023-12-19T12:34:46Z)
• Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs [6.847114270274019]
  We show how to generate labeled data that can be used to train a downstream NLP model. We introduce Fabricator, an open-source Python toolkit for NLP generation.
  arXiv  Detail & Related papers  (2023-09-18T08:45:47Z)
• MLLM-DataEngine: An Iterative Refinement Approach for MLLM [62.30753425449056]
  We propose a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results. For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data. For quality, we resort to GPT-4 to generate high-quality data with each given data type.
  arXiv  Detail & Related papers  (2023-08-25T01:41:04Z)
• WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation [101.00109827301235]
  We introduce a novel paradigm for dataset creation based on human and machine collaboration. We use dataset cartography to automatically identify examples that demonstrate challenging reasoning patterns, and instruct GPT-3 to compose new examples with similar patterns. The resulting dataset, WANLI, consists of 108,357 natural language inference (NLI) examples that present unique empirical strengths.
  arXiv  Detail & Related papers  (2022-01-16T03:13:49Z)
• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP [88.65488361532158]
  dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks. Data augmentation methods have been explored as a means of improving data efficiency in NLP. We provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting.
  arXiv  Detail & Related papers  (2021-06-14T15:27:22Z)
• The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures [0.0]
  Natural Language Processing models have achieved phenomenal success in linguistic and semantic tasks. Recent NLP architectures have utilized concepts of transfer learning, pruning, quantization, and knowledge distillation to achieve moderate model sizes. Knowledge Retrievers have been built to extricate explicit data documents from a large corpus of databases with greater efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-03-23T22:38:20Z)
• DQI: Measuring Data Quality in NLP [22.54066527822898]
  We introduce a generic formula for Data Quality Index (DQI) to help dataset creators create datasets free of unwanted biases. We show that models trained on the renovated SNLI dataset generalize better to out of distribution tasks.
  arXiv  Detail & Related papers  (2020-05-02T12:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.