FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

• URL: http://arxiv.org/abs/2402.10986v3
• Date: Fri, 14 Jun 2024 13:26:47 GMT
• Title: FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models
• Authors: Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, Muhammad Abdul-Mageed,
• Abstract summary: FinTral is a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model. We enhance FinTral with domain-specific pretraining, instruction fine-tuning, and RLAIF training. Our FinTral model trained with direct preference optimization employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R, demonstrates an exceptional zero-shot performance.
• Score: 18.280762424107408
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We introduce FinTral, a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model and tailored for financial analysis. FinTral integrates textual, numerical, tabular, and image data. We enhance FinTral with domain-specific pretraining, instruction fine-tuning, and RLAIF training by exploiting a large collection of textual and visual datasets we curate for this work. We also introduce an extensive benchmark featuring nine tasks and 25 datasets for evaluation, including hallucinations in the financial domain. Our FinTral model trained with direct preference optimization employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R, demonstrates an exceptional zero-shot performance. It outperforms ChatGPT-3.5 in all tasks and surpasses GPT-4 in five out of nine tasks, marking a significant advancement in AI-driven financial technology. We also demonstrate that FinTral has the potential to excel in real-time analysis and decision-making in diverse financial contexts. The GitHub repository for FinTral is available at \url{https://github.com/UBC-NLP/fintral}.

Related papers

• Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications [90.67346776473241]
  Large language models (LLMs) have advanced financial applications, yet they often lack sufficient financial knowledge and struggle with tasks involving multi-modal inputs like tables and time series data. We introduce textitOpen-FinLLMs, a series of Financial LLMs that embed comprehensive financial knowledge into text, tables, and time-series data. We also present FinLLaVA, a multimodal LLM trained with 1.43M image-text instructions to handle complex financial data types.
  arXiv  Detail & Related papers  (2024-08-20T16:15:28Z)
• SNFinLLM: Systematic and Nuanced Financial Domain Adaptation of Chinese Large Language Models [6.639972934967109]
  Large language models (LLMs) have become powerful tools for advancing natural language processing applications in the financial industry. We propose a novel large language model specifically designed for the Chinese financial domain, named SNFinLLM. SNFinLLM excels in domain-specific tasks such as answering questions, summarizing financial research reports, analyzing sentiment, and executing financial calculations.
  arXiv  Detail & Related papers  (2024-08-05T08:24:24Z)
• AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework [48.3060010653088]
  We release AlphaFin datasets, combining traditional research datasets, real-time financial data, and handwritten chain-of-thought (CoT) data. We then use AlphaFin datasets to benchmark a state-of-the-art method, called Stock-Chain, for effectively tackling the financial analysis task.
  arXiv  Detail & Related papers  (2024-03-19T09:45:33Z)
• FinBen: A Holistic Financial Benchmark for Large Language Models [75.09474986283394]
  FinBen is the first extensive open-source evaluation benchmark, including 36 datasets spanning 24 financial tasks. FinBen offers several key innovations: a broader range of tasks and datasets, the first evaluation of stock trading, novel agent and Retrieval-Augmented Generation (RAG) evaluation, and three novel open-source evaluation datasets for text summarization, question answering, and stock trading.
  arXiv  Detail & Related papers  (2024-02-20T02:16:16Z)
• A Survey of Large Language Models in Finance (FinLLMs) [10.195778659105626]
  Large Language Models (LLMs) have shown remarkable capabilities across a wide variety of Natural Language Processing (NLP) tasks. This survey provides a comprehensive overview of FinLLMs, including their history, techniques, performance, and opportunities and challenges. To support AI research in finance, we compile a collection of accessible datasets and evaluation benchmarks on GitHub.
  arXiv  Detail & Related papers  (2024-02-04T02:06:57Z)
• DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning [74.99318727786337]
  We propose Multiple Experts Fine-tuning Framework to build a financial large language model (LLM) We build a financial instruction-tuning dataset named DISC-FIN-SFT, including instruction samples of four categories (consulting, NLP tasks, computing and retrieval-augmented generation) Evaluations conducted on multiple benchmarks demonstrate that our model performs better than baseline models in various financial scenarios.
  arXiv  Detail & Related papers  (2023-10-23T11:33:41Z)
• CFGPT: Chinese Financial Assistant with Large Language Model [21.54229667774752]
  We present a Chinese Financial Generative Pre-trained Transformer framework, named CFGPT. CFData comprises both a pre-training dataset and a supervised fine-tuning dataset. CFLLM is trained on CFData in two stage, continued pre-training and supervised fine-tuning.
  arXiv  Detail & Related papers  (2023-09-19T14:34:01Z)
• FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis [15.20897845057384]
  FinVis-GPT is a novel multimodal large language model (LLM) specifically designed for financial chart analysis. The proposed FinVis-GPT serves as a pioneering effort in utilizing multimodal LLMs in the finance domain.
  arXiv  Detail & Related papers  (2023-07-31T07:44:15Z)
• PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance [63.51545277822702]
  PIXIU is a comprehensive framework including the first financial large language model (LLMs) based on fine-tuning LLaMA with instruction data. We propose FinMA by fine-tuning LLaMA with the constructed dataset to be able to follow instructions for various financial tasks. We conduct a detailed analysis of FinMA and several existing LLMs, uncovering their strengths and weaknesses in handling critical financial tasks.
  arXiv  Detail & Related papers  (2023-06-08T14:20:29Z)
• Dynamic Datasets and Market Environments for Financial Reinforcement Learning [68.11692837240756]
  FinRL-Meta is a library that processes dynamic datasets from real-world markets into gym-style market environments. We provide examples and reproduce popular research papers as stepping stones for users to design new trading strategies. We also deploy the library on cloud platforms so that users can visualize their own results and assess the relative performance.
  arXiv  Detail & Related papers  (2023-04-25T22:17:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.