Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generation

• URL: http://arxiv.org/abs/2309.08793v2
• Date: Wed, 1 May 2024 21:44:34 GMT
• Title: Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generation
• Authors: Aman Rangapur, Haoran Wang, Ling Jian, Kai Shu,
• Abstract summary: Fin-Fact is a benchmark dataset for multimodal fact-checking within the financial domain. It includes professional fact-checker annotations and justifications, providing expertise and credibility.
• Score: 26.573578326262307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fact-checking in financial domain is under explored, and there is a shortage of quality dataset in this domain. In this paper, we propose Fin-Fact, a benchmark dataset for multimodal fact-checking within the financial domain. Notably, it includes professional fact-checker annotations and justifications, providing expertise and credibility. With its multimodal nature encompassing both textual and visual content, Fin-Fact provides complementary information sources to enhance factuality analysis. Its primary objective is combating misinformation in finance, fostering transparency, and building trust in financial reporting and news dissemination. By offering insightful explanations, Fin-Fact empowers users, including domain experts and end-users, to understand the reasoning behind fact-checking decisions, validating claim credibility, and fostering trust in the fact-checking process. The Fin-Fact dataset, along with our experimental codes is available at https://github.com/IIT-DM/Fin-Fact/.

Related papers

• AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework [48.3060010653088]
  We release AlphaFin datasets, combining traditional research datasets, real-time financial data, and handwritten chain-of-thought (CoT) data. We then use AlphaFin datasets to benchmark a state-of-the-art method, called Stock-Chain, for effectively tackling the financial analysis task.
  arXiv  Detail & Related papers  (2024-03-19T09:45:33Z)
• FinBen: A Holistic Financial Benchmark for Large Language Models [75.09474986283394]
  FinBen is the first extensive open-source evaluation benchmark, including 36 datasets spanning 24 financial tasks. FinBen offers several key innovations: a broader range of tasks and datasets, the first evaluation of stock trading, novel agent and Retrieval-Augmented Generation (RAG) evaluation, and three novel open-source evaluation datasets for text summarization, question answering, and stock trading.
  arXiv  Detail & Related papers  (2024-02-20T02:16:16Z)
• FinEntity: Entity-level Sentiment Classification for Financial Texts [15.467477195487763]
  In the financial domain, conducting entity-level sentiment analysis is crucial for accurately assessing the sentiment directed toward a specific financial entity. We introduce an entity-level sentiment classification dataset, called textbfFinEntity, that annotates financial entity spans and their sentiment in financial news.
  arXiv  Detail & Related papers  (2023-10-19T01:38:40Z)
• PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance [63.51545277822702]
  PIXIU is a comprehensive framework including the first financial large language model (LLMs) based on fine-tuning LLaMA with instruction data. We propose FinMA by fine-tuning LLaMA with the constructed dataset to be able to follow instructions for various financial tasks. We conduct a detailed analysis of FinMA and several existing LLMs, uncovering their strengths and weaknesses in handling critical financial tasks.
  arXiv  Detail & Related papers  (2023-06-08T14:20:29Z)
• Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs [48.87627426640621]
  This research focuses on harnessing the potential of Large Language Models to comprehend critical information from financial reports. We propose an Automated Financial Information Extraction framework that enhances LLMs' ability to comprehend and extract information from financial reports. Our framework is effectively validated on GPT-3.5 and GPT-4, yielding average accuracy increases of 53.94% and 33.77%, respectively.
  arXiv  Detail & Related papers  (2023-05-24T10:35:58Z)
• Measuring Consistency in Text-based Financial Forecasting Models [10.339586273664725]
  FinTrust is an evaluation tool that assesses logical consistency in financial text. We show that the consistency of state-of-the-art NLP models for financial forecasting is poor. Our analysis of the performance degradation caused by meaning-preserving alternations suggests that current text-based methods are not suitable for robustly predicting market information.
  arXiv  Detail & Related papers  (2023-05-15T10:32:26Z)
• FinQA: A Dataset of Numerical Reasoning over Financial Data [52.7249610894623]
  We focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. We propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge.
  arXiv  Detail & Related papers  (2021-09-01T00:08:14Z)
• Fact Check: Analyzing Financial Events from Multilingual News Sources [22.504723681328507]
  We propose FactCheck in finance, a web-based news aggregator with deep learning models. A web interface is provided to examine the credibility of news articles using a transformer-based fact-checker. The performance of the fact checker is evaluated using a dataset related to merger and acquisition (M&A) events.
  arXiv  Detail & Related papers  (2021-06-29T10:05:47Z)
• Insights into Fairness through Trust: Multi-scale Trust Quantification for Financial Deep Learning [94.65749466106664]
  A fundamental aspect of fairness that has not been explored in financial deep learning is the concept of trust. We conduct multi-scale trust quantification on a deep neural network for the purpose of credit card default prediction.
  arXiv  Detail & Related papers  (2020-11-03T19:05:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.