Leveraging Large Language Models to Democratize Access to Costly Financial Datasets for Academic Research

• URL: http://arxiv.org/abs/2412.02065v1
• Date: Tue, 03 Dec 2024 00:59:56 GMT
• Title: Leveraging Large Language Models to Democratize Access to Costly Financial Datasets for Academic Research
• Authors: Julian Junyan Wang, Victor Xiaoqi Wang,
• Abstract summary: We develop and evaluate a novel methodology using GPT-4o-mini within a Retrieval-Augmented Generation (RAG) framework to collect data from corporate disclosures. Our approach achieves human-level accuracy in collecting CEO pay ratios from approximately 10,000 proxy statements and Critical Audit Matters (CAMs) from more than 12,000 10-K filings. This stands in stark contrast to the hundreds of hours needed for manual collection or the thousands of dollars required for commercial database subscriptions.
• Score: 0.0
• License:
• Abstract: Unequal access to costly datasets essential for empirical research has long hindered researchers from disadvantaged institutions, limiting their ability to contribute to their fields and advance their careers. Recent breakthroughs in Large Language Models (LLMs) have the potential to democratize data access by automating data collection from unstructured sources. We develop and evaluate a novel methodology using GPT-4o-mini within a Retrieval-Augmented Generation (RAG) framework to collect data from corporate disclosures. Our approach achieves human-level accuracy in collecting CEO pay ratios from approximately 10,000 proxy statements and Critical Audit Matters (CAMs) from more than 12,000 10-K filings, with LLM processing times of 9 and 40 minutes respectively, each at a cost under $10. This stands in stark contrast to the hundreds of hours needed for manual collection or the thousands of dollars required for commercial database subscriptions. To foster a more inclusive research community by empowering researchers with limited resources to explore new avenues of inquiry, we share our methodology and the resulting datasets.

Related papers

• SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval [30.269970599368815]
  We extend the TREC Deep Learning Track (DL) test collection via additional language model synthetic labels to enable researchers to test and evaluate their search systems at a large scale. Specifically, such a test collection includes more than 1,900 test queries from the previous years of tracks. We compare system evaluation with past human labels from past years and find that our synthetically created large-scale test collection can lead to highly correlated system rankings.
  arXiv  Detail & Related papers  (2024-08-29T07:20:56Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• Data-Centric AI in the Age of Large Language Models [51.20451986068925]
  This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs) We make the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization.
  arXiv  Detail & Related papers  (2024-06-20T16:34:07Z)
• A Survey on Data Selection for Language Models [148.300726396877]
  Data selection methods aim to determine which data points to include in a training dataset. Deep learning is mostly driven by empirical evidence and experimentation on large-scale data is expensive. Few organizations have the resources for extensive data selection research.
  arXiv  Detail & Related papers  (2024-02-26T18:54:35Z)
• Query of CC: Unearthing Large Scale Domain-Specific Knowledge from Public Corpora [104.16648246740543]
  We propose an efficient data collection method based on large language models. The method bootstraps seed information through a large language model and retrieves related data from public corpora. It not only collects knowledge-related data for specific domains but unearths the data with potential reasoning procedures.
  arXiv  Detail & Related papers  (2024-01-26T03:38:23Z)
• Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking [3.9412826185755017]
  Few-shot and large language models (LLMs) can perform effectively with just 1-5 examples per class. Our work addresses the performance-cost trade-offs of these methods over the Banking77 financial intent detection dataset. To inspire future research, we provide a human expert's curated subset of Banking77, along with extensive error analysis.
  arXiv  Detail & Related papers  (2023-11-10T15:10:36Z)
• CSPRD: A Financial Policy Retrieval Dataset for Chinese Stock Market [61.59326951366202]
  We propose a new task, policy retrieval, by introducing the Chinese Stock Policy Retrieval dataset (CSPRD) CSPRD provides 700+ passages labeled by experienced experts with relevant articles from 10k+ entries in our collected Chinese policy corpus. Our best performing baseline achieves 56.1% MRR@10, 28.5% NDCG@10, 37.5% Recall@10 and 80.6% Precision@10 on dev set.
  arXiv  Detail & Related papers  (2023-09-08T15:40:54Z)
• Optimizing Data Collection for Machine Learning [87.37252958806856]
  Modern deep learning systems require huge data sets to achieve impressive performance. Over-collecting data incurs unnecessary present costs, while under-collecting may incur future costs and delay. We propose a new paradigm for modeling the data collection as a formal optimal data collection problem.
  arXiv  Detail & Related papers  (2022-10-03T21:19:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.