CFGPT: Chinese Financial Assistant with Large Language Model
- URL: http://arxiv.org/abs/2309.10654v2
- Date: Fri, 22 Sep 2023 09:52:07 GMT
- Title: CFGPT: Chinese Financial Assistant with Large Language Model
- Authors: Jiangtong Li, Yuxuan Bian, Guoxuan Wang, Yang Lei, Dawei Cheng, Zhijun
Ding and Changjun Jiang
- Abstract summary: We present a Chinese Financial Generative Pre-trained Transformer framework, named CFGPT.
CFData comprises both a pre-training dataset and a supervised fine-tuning dataset.
CFLLM is trained on CFData in two stage, continued pre-training and supervised fine-tuning.
- Score: 21.54229667774752
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models (LLMs) have demonstrated great potential in natural
language processing tasks within the financial domain. In this work, we present
a Chinese Financial Generative Pre-trained Transformer framework, named CFGPT,
which includes a dataset~(CFData) for pre-training and supervised fine-tuning,
a financial LLM~(CFLLM) to adeptly manage financial texts, and a deployment
framework~(CFAPP) designed to navigate real-world financial applications. The
CFData comprising both a pre-training dataset and a supervised fine-tuning
dataset, where the pre-training dataset collates Chinese financial data and
analytics, alongside a smaller subset of general-purpose text with 584M
documents and 141B tokens in total, and the supervised fine-tuning dataset is
tailored for six distinct financial tasks, embodying various facets of
financial analysis and decision-making with 1.5M instruction pairs and 1.5B
tokens in total. The CFLLM, which is based on InternLM-7B to balance the model
capability and size, is trained on CFData in two stage, continued pre-training
and supervised fine-tuning. The CFAPP is centered on large language models
(LLMs) and augmented with additional modules to ensure multifaceted
functionality in real-world application. Our codes are released at
https://github.com/TongjiFinLab/CFGPT.
Related papers
- CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications [10.225210627594894]
This paper presents our solution to IJCAI-2024 FinLLM challenge, investigating the capabilities of LLMs within three critical areas of financial tasks.
Financial classification, financial text summarization, and single stock trading are investigated.
Our approach aims to tackle these diverse tasks in a comprehensive and integrated manner, showcasing LLMs' capacity to address diverse and complex financial tasks with improved accuracy and decision-making capabilities.
arXiv Detail & Related papers (2024-07-02T05:04:13Z) - NumLLM: Numeric-Sensitive Large Language Model for Chinese Finance [15.662858834827444]
FinLLMs exhibit unsatisfactory performance in understanding financial text when numeric variables are involved in questions.
We propose numeric-sensitive large language model (NumLLM) for Chinese finance.
Experiments on financial question-answering benchmark show that NumLLM can boost the performance of the foundation model.
arXiv Detail & Related papers (2024-05-01T15:17:27Z) - Tele-FLM Technical Report [96.19923831660266]
We introduce Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model.
It features a stable, efficient pre-training paradigm and enhanced factual judgment capabilities.
It is comparable to strong open-sourced models that involve larger pre-training FLOPs, such as Llama2-70B and DeepSeek-67B.
arXiv Detail & Related papers (2024-04-25T14:34:47Z) - AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework [48.3060010653088]
We release AlphaFin datasets, combining traditional research datasets, real-time financial data, and handwritten chain-of-thought (CoT) data.
We then use AlphaFin datasets to benchmark a state-of-the-art method, called Stock-Chain, for effectively tackling the financial analysis task.
arXiv Detail & Related papers (2024-03-19T09:45:33Z) - FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models [18.280762424107408]
FinTral is a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model.
We enhance FinTral with domain-specific pretraining, instruction fine-tuning, and RLAIF training.
Our FinTral model trained with direct preference optimization employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R, demonstrates an exceptional zero-shot performance.
arXiv Detail & Related papers (2024-02-16T05:05:12Z) - D\'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs
Between Spanish and English [67.48541936784501]
Tois'on de Oro is the first framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English.
We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks.
We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks.
arXiv Detail & Related papers (2024-02-12T04:50:31Z) - Unleashing the Power of Pre-trained Language Models for Offline
Reinforcement Learning [54.682106515794864]
offline reinforcement learning (RL) aims to find a near-optimal policy using pre-collected datasets.
This paper introduces $textbfLanguage Models for $textbfMo$tion Control ($textbfLaMo$), a general framework based on Decision Transformers to use pre-trained Language Models (LMs) for offline RL.
Empirical results indicate $textbfLaMo$ achieves state-of-the-art performance in sparse-reward tasks.
arXiv Detail & Related papers (2023-10-31T16:24:17Z) - DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple
Experts Fine-tuning [74.99318727786337]
We propose Multiple Experts Fine-tuning Framework to build a financial large language model (LLM)
We build a financial instruction-tuning dataset named DISC-FIN-SFT, including instruction samples of four categories (consulting, NLP tasks, computing and retrieval-augmented generation)
Evaluations conducted on multiple benchmarks demonstrate that our model performs better than baseline models in various financial scenarios.
arXiv Detail & Related papers (2023-10-23T11:33:41Z) - PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark
for Finance [63.51545277822702]
PIXIU is a comprehensive framework including the first financial large language model (LLMs) based on fine-tuning LLaMA with instruction data.
We propose FinMA by fine-tuning LLaMA with the constructed dataset to be able to follow instructions for various financial tasks.
We conduct a detailed analysis of FinMA and several existing LLMs, uncovering their strengths and weaknesses in handling critical financial tasks.
arXiv Detail & Related papers (2023-06-08T14:20:29Z) - BloombergGPT: A Large Language Model for Finance [42.73350054822628]
We present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data.
We construct a 363 billion token dataset based on Bloomberg's extensive data sources, augmented with 345 billion tokens from general purpose datasets.
Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins.
arXiv Detail & Related papers (2023-03-30T17:30:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.