DeepDiveAI: Identifying AI Related Documents in Large Scale Literature Data

• URL: http://arxiv.org/abs/2408.12871v3
• Date: Tue, 8 Oct 2024 07:21:57 GMT
• Title: DeepDiveAI: Identifying AI Related Documents in Large Scale Literature Data
• Authors: Zhou Xiaochen, Liang Xingzhou, Zou Hui, Lu Yi, Qu Jingjing,
• Abstract summary: The dataset was created using an advanced Long Short-Term Memory (LSTM) model trained on a binary classification task. The model was trained and validated on a vast dataset, achieving high accuracy, precision, recall, and F1-score. The resulting DeepDelveAI dataset comprises over 9.4 million AI-related papers published since Dartmouth Conference, from 1956 to 2024.
• Score: 4.870043547158868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents DeepDiveAI, a comprehensive dataset specifically curated to identify AI-related research papers from a large-scale academic literature database. The dataset was created using an advanced Long Short-Term Memory (LSTM) model trained on a binary classification task to distinguish between AI-related and non-AI-related papers. The model was trained and validated on a vast dataset, achieving high accuracy, precision, recall, and F1-score. The resulting DeepDelveAI dataset comprises over 9.4 million AI-related papers published since Dartmouth Conference, from 1956 to 2024, providing a crucial resource for analyzing trends, thematic developments, and the evolution of AI research across various disciplines.

Related papers

• Closing the Data Loop: Using OpenDataArena to Engineer Superior Training Datasets [46.480867560675584]
  We propose a paradigm shift from ad-hoc curation to a closed-loop dataset engineering framework using OpenDataArena (ODA)<n>We instantiate this methodology through two new datasets: textbfODA-Math460-k, a specialized mathematics reasoning dataset that utilizes a novel two-stage difficulty-aware pipeline to achieve State-of-the-Art (SOTA) results on benchmarks such as AIME and HMMT, and textbfODA-Mixture (100k & 500k), a series of multi-domain instruction datasets built via an Anchor-and-
  arXiv  Detail & Related papers  (2025-12-30T17:46:38Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• AiEDA: An Open-Source AI-Aided Design Library for Design-to-Vector [13.443294857996852]
  Current AI for EDA (AI-EDA) infrastructures remain fragmented, lacking comprehensive solutions for the entire data pipeline from design execution to AI integration.<n>This work introduces a unified open-source library for EDA (AiEDA) that addresses these issues.<n>AiEDA integrates multiple design-to-vector data representation techniques that transform diverse chip design data into universal multi-level vector representations.
  arXiv  Detail & Related papers  (2025-11-08T03:14:26Z)
• Standards in the Preparation of Biomedical Research Metadata: A Bridge2AI Perspective [1.0389904886733017]
  Bridge2AI has defined the criteria a dataset may possess to render it AI-ready.<n>These criteria include FAIRness, provenance, degree of characterization, explainability, sustainability, and computability.<n>This report assesses the state of metadata creation and standardization in the Bridge2AI Grand Challenges.
  arXiv  Detail & Related papers  (2025-09-12T17:38:46Z)
• Large Language Models and Synthetic Data for Monitoring Dataset Mentions in Research Papers [0.0]
  This paper presents a machine learning framework that automates dataset mention detection across research domains. We employ zero-shot extraction from research papers, an LLM-as-a-Judge for quality assessment, and a reasoning agent for refinement to generate a weakly supervised synthetic dataset. At inference, a ModernBERT-based classifier efficiently filters dataset mentions, reducing computational overhead while maintaining high recall.
  arXiv  Detail & Related papers  (2025-02-14T16:16:02Z)
• TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation [50.23504065567638]
  This paper introduces textbfTD3, a novel textbfDataset textbfDistillation method within a meta-learning framework. TD3 distills a fully expressive emphsynthetic sequence summary from original data. An augmentation technique allows the learner to closely fit the synthetic summary, ensuring an accurate update of it in the emphouter-loop.
  arXiv  Detail & Related papers  (2025-02-05T03:13:25Z)
• DSAI: Unbiased and Interpretable Latent Feature Extraction for Data-Centric AI [24.349800949355465]
  Large language models (LLMs) often struggle to objectively identify latent characteristics in large datasets. We propose Data Scientist AI (DSAI), a framework that enables unbiased and interpretable feature extraction.
  arXiv  Detail & Related papers  (2024-12-09T08:47:05Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation [51.2289822267563]
  We propose a method for generating synthetic datasets, given a small number of user-written few-shots that demonstrate the task to be performed.<n>We demonstrate that CRAFT can efficiently generate large-scale task-specific training datasets for four diverse tasks: biology, medicine, and commonsense question-answering (QA)<n>Our experiments show that CRAFT-based models outperform or match general LLMs on QA tasks, while exceeding models trained on human-curated summarization data by 46 preference points.
  arXiv  Detail & Related papers  (2024-09-03T17:54:40Z)
• Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting [36.31269406067809]
  We argue that data-centric AI is essential for training AI models, particularly for transformer-based TSF models efficiently. We review the previous research works from a data-centric AI perspective and we intend to lay the foundation work for the future development of transformer-based architecture and data-centric AI.
  arXiv  Detail & Related papers  (2024-07-29T08:27:21Z)
• Data-Centric AI in the Age of Large Language Models [51.20451986068925]
  This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs) We make the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization.
  arXiv  Detail & Related papers  (2024-06-20T16:34:07Z)
• StyloAI: Distinguishing AI-Generated Content with Stylometric Analysis [0.0]
  This study proposes StyloAI, a data-driven model that uses 31 stylometric features to identify AI-generated texts. StyloAI achieves accuracy rates of 81% and 98% on the test set of the AuTextification dataset and the Education dataset, respectively.
  arXiv  Detail & Related papers  (2024-05-16T14:28:01Z)
• Data Readiness for AI: A 360-Degree Survey [0.9343816282846432]
  Poor quality data produces inaccurate and ineffective AI models. Numerous R&D efforts have been spent on improving data quality. We propose a taxonomy of data readiness for AI (DRAI) metrics for structured and unstructured datasets.
  arXiv  Detail & Related papers  (2024-04-08T15:19:57Z)
• DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation [83.30006900263744]
  Data analysis is a crucial analytical process to generate in-depth studies and conclusive insights. We propose to automatically generate high-quality answer annotations leveraging the code-generation capabilities of LLMs. Our DACO-RL algorithm is evaluated by human annotators to produce more helpful answers than SFT model in 57.72% cases.
  arXiv  Detail & Related papers  (2024-03-04T22:47:58Z)
• Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization [3.6393183544320236]
  Speech recognition has become an important challenge when using deep learning (DL) It requires large-scale training datasets and high computational and storage resources. Deep transfer learning (DTL) has been introduced to overcome these issues.
  arXiv  Detail & Related papers  (2023-04-27T21:08:05Z)
• Data-centric Artificial Intelligence: A Survey [47.24049907785989]
  Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
  arXiv  Detail & Related papers  (2023-03-17T17:44:56Z)
• Research Trends and Applications of Data Augmentation Algorithms [77.34726150561087]
  We identify the main areas of application of data augmentation algorithms, the types of algorithms used, significant research trends, their progression over time and research gaps in data augmentation literature. We expect readers to understand the potential of data augmentation, as well as identify future research directions and open questions within data augmentation research.
  arXiv  Detail & Related papers  (2022-07-18T11:38:32Z)
• Unsupervised Domain Adaptive Learning via Synthetic Data for Person Re-identification [101.1886788396803]
  Person re-identification (re-ID) has gained more and more attention due to its widespread applications in video surveillance. Unfortunately, the mainstream deep learning methods still need a large quantity of labeled data to train models. In this paper, we develop a data collector to automatically generate synthetic re-ID samples in a computer game, and construct a data labeler to simultaneously annotate them.
  arXiv  Detail & Related papers  (2021-09-12T15:51:41Z)
• Paradigm selection for Data Fusion of SAR and Multispectral Sentinel data applied to Land-Cover Classification [63.072664304695465]
  In this letter, four data fusion paradigms, based on Convolutional Neural Networks (CNNs) are analyzed and implemented. The goals are to provide a systematic procedure for choosing the best data fusion framework, resulting in the best classification results. The procedure has been validated for land-cover classification but it can be transferred to other cases.
  arXiv  Detail & Related papers  (2021-06-18T11:36:54Z)
• Data Augmentation for Abstractive Query-Focused Multi-Document Summarization [129.96147867496205]
  We present two QMDS training datasets, which we construct using two data augmentation methods. These two datasets have complementary properties, i.e., QMDSCNN has real summaries but queries are simulated, while QMDSIR has real queries but simulated summaries. We build end-to-end neural network models on the combined datasets that yield new state-of-the-art transfer results on DUC datasets.
  arXiv  Detail & Related papers  (2021-03-02T16:57:01Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.