CSPRD: A Financial Policy Retrieval Dataset for Chinese Stock Market

• URL: http://arxiv.org/abs/2309.04389v2
• Date: Mon, 11 Sep 2023 05:19:16 GMT
• Title: CSPRD: A Financial Policy Retrieval Dataset for Chinese Stock Market
• Authors: Jinyuan Wang, Hai Zhao, Zhong Wang, Zeyang Zhu, Jinhao Xie, Yong Yu, Yongjian Fei, Yue Huang and Dawei Cheng
• Abstract summary: We propose a new task, policy retrieval, by introducing the Chinese Stock Policy Retrieval dataset (CSPRD) CSPRD provides 700+ passages labeled by experienced experts with relevant articles from 10k+ entries in our collected Chinese policy corpus. Our best performing baseline achieves 56.1% MRR@10, 28.5% NDCG@10, 37.5% Recall@10 and 80.6% Precision@10 on dev set.
• Score: 61.59326951366202
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, great advances in pre-trained language models (PLMs) have sparked considerable research focus and achieved promising performance on the approach of dense passage retrieval, which aims at retrieving relative passages from massive corpus with given questions. However, most of existing datasets mainly benchmark the models with factoid queries of general commonsense, while specialised fields such as finance and economics remain unexplored due to the deficiency of large-scale and high-quality datasets with expert annotations. In this work, we propose a new task, policy retrieval, by introducing the Chinese Stock Policy Retrieval Dataset (CSPRD), which provides 700+ prospectus passages labeled by experienced experts with relevant articles from 10k+ entries in our collected Chinese policy corpus. Experiments on lexical, embedding and fine-tuned bi-encoder models show the effectiveness of our proposed CSPRD yet also suggests ample potential for improvement. Our best performing baseline achieves 56.1% MRR@10, 28.5% NDCG@10, 37.5% Recall@10 and 80.6% Precision@10 on dev set.

Related papers

• A Comprehensive Survey of Datasets, Theories, Variants, and Applications in Direct Preference Optimization [52.42860559005861]
  Direct Preference Optimization (DPO) has emerged as a promising approach for alignment. Despite DPO's various advancements and inherent limitations, an in-depth review of these aspects is currently lacking in the literature.
  arXiv  Detail & Related papers  (2024-10-21T02:27:24Z)
• Entity Extraction from High-Level Corruption Schemes via Large Language Models [4.820586736502356]
  This article proposes a new micro-benchmark dataset for algorithms and models that identify individuals and organizations in news articles. Experimental efforts are also reported, using this dataset, to identify individuals and organizations in financial-crime-related articles.
  arXiv  Detail & Related papers  (2024-09-05T10:27:32Z)
• Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning [70.22819290458581]
  Reinforcement learning with human feedback (RLHF) is a widely adopted approach in current large language model pipelines. Our approach introduces two key innovations: (1) on-policy query to avoid OOD and imbalance issues in seed data, and (2) active learning to select the most informative data for preference queries.
  arXiv  Detail & Related papers  (2024-07-02T10:09:19Z)
• Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking [59.87055275344965]
  Job-SDF is a dataset designed to train and benchmark job-skill demand forecasting models. Based on 10.35 million public job advertisements collected from major online recruitment platforms in China between 2021 and 2023. Our dataset uniquely enables evaluating skill demand forecasting models at various granularities, including occupation, company, and regional levels.
  arXiv  Detail & Related papers  (2024-06-17T07:22:51Z)
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
  We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process. We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals. The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
  arXiv  Detail & Related papers  (2024-05-01T11:10:24Z)
• DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries [2.4816250611120547]
  We propose a novel task, Scientific DOcument Retrieval using Multi-level Aspect-based quEries (DORIS-MAE) For each complex query, we assembled a collection of 100 relevant documents and produced annotated relevance scores for ranking them. Anno-GPT is a framework for validating the performance of Large Language Models (LLMs) on expert-level dataset annotation tasks.
  arXiv  Detail & Related papers  (2023-10-07T03:25:06Z)
• A Comparative Study of Hierarchical Risk Parity Portfolio and Eigen Portfolio on the NIFTY 50 Stocks [1.5773159234875098]
  This paper presents a systematic approach to portfolio optimization using two approaches, the hierarchical risk parity algorithm and the Eigen portfolio on seven sectors of the Indian stock market. The backtesting results of the portfolios indicate that the performance of the HRP portfolio is superior to that of its counterpart on both training and test data for the majority of the sectors studied.
  arXiv  Detail & Related papers  (2022-10-03T14:51:24Z)
• Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies [74.01792675564218]
  We develop a data augmentation framework based on ensembling retriever models that captures relevant text segments from unlabeled policy documents. To improve the diversity and quality of the augmented data, we leverage multiple pre-trained language models (LMs) and cascade them with noise reduction filter models. Using our augmented data on the PrivacyQA benchmark, we elevate the existing baseline by a large margin (10% F1) and achieve a new state-of-the-art F1 score of 50%.
  arXiv  Detail & Related papers  (2022-04-19T15:45:23Z)
• Whose AI Dream? In search of the aspiration in data annotation [12.454034525520497]
  This paper investigates the work practices concerning data annotation as performed in the industry, in India. Previous investigations have largely focused on annotator subjectivity, bias and efficiency. Our results show that the work of annotators is dictated by the interests, priorities and values of others above their station.
  arXiv  Detail & Related papers  (2022-03-21T06:28:54Z)
• Context-NER : Contextual Phrase Generation at Scale [4.7947627446578025]
  We introduce CONTEXT-NER, a task that aims to generate relevant context for entities in a sentence. We present the EDGAR10-Q dataset, containing 1M sentences, 2.8M entities, and an average of 35 tokens per sentence. We find that T5-large, when pre-finetuned on EDGAR10-Q, achieve SOTA results on downstream finance tasks such as Headline, FPB, and FiQA SA, outperforming vanilla version by 10.81 points.
  arXiv  Detail & Related papers  (2021-09-16T16:10:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.