Case-Based Reasoning Approach for Solving Financial Question Answering

• URL: http://arxiv.org/abs/2405.13044v1
• Date: Sat, 18 May 2024 10:06:55 GMT
• Title: Case-Based Reasoning Approach for Solving Financial Question Answering
• Authors: Yikyung Kim, Jay-Yoon Lee,
• Abstract summary: FinQA introduced a numerical reasoning dataset for financial documents. We propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR) Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information.
• Score: 5.10832476049103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Measuring a machine's understanding of human language often involves assessing its reasoning skills, i.e. logical process of deriving answers to questions. While recent language models have shown remarkable proficiency in text based tasks, their efficacy in complex reasoning problems involving heterogeneous information such as text, tables, and numbers remain uncertain. Addressing this gap, FinQA introduced a numerical reasoning dataset for financial documents and simultaneously proposed a program generation approach . Our investigation reveals that half of the errors (48%) stem from incorrect operations being generated. To address this issue, we propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR), an artificial intelligence paradigm that provides problem solving guidance by offering similar cases (i.e. similar questions and corresponding logical programs). Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information. Through experiments on the FinQA dataset, we demonstrate competitive performance of our approach and additionally show that by expanding case repository, we can help solving complex multi step programs which FinQA showed weakness of.

Related papers

• RM-PoT: Reformulating Mathematical Problems and Solving via Program of Thoughts [13.07180561863778]
  We propose a three-stage framework that integrates problem reformulation (RM), code-aided reasoning (PoT) and domain-aware few-shot learning. Our approach first reformulates the input problem into diverse surface forms to reduce structural bias, then retrieves five semantically aligned examples to provide contextual guidance.
  arXiv  Detail & Related papers  (2025-02-18T06:54:32Z)
• PEA: Enhancing LLM Performance on Computational-Reasoning Tasks [21.13926189404758]
  This study introduces a formal approach to describe and solve a class of important reasoning tasks termed computational reasoning problems. The framework decomposes these problems into predicate and enumeration components, using LLMs to synthesize programs based on specified predicates, enumeration, and aggregation rules. Empirical evaluation reveals that PEA substantially enhances the performance of underlying models on benchmark computational problems, yielding an average accuracy improvement of approximately $50%$, coupled with increased efficiency.
  arXiv  Detail & Related papers  (2025-02-16T00:27:05Z)
• Instantiation-based Formalization of Logical Reasoning Tasks using Language Models and Logical Solvers [4.897782942277061]
  We introduce Semantic Self-Verification (SSV), a novel approach to accurately formulate the reasoning problem from natural language to the formal language of the solver. SSV uses a consistency-based approach to produce strong abstract formalizations of problems using concrete instantiations that are generated by the model and verified by the solver. We propose such *near-certain reasoning* as a new approach to reduce the need for manual verification in many cases, taking us closer to more dependable and autonomous AI reasoning systems.
  arXiv  Detail & Related papers  (2025-01-28T14:04:49Z)
• Advancing Algorithmic Approaches to Probabilistic Argumentation under the Constellation Approach [0.0]
  We develop an algorithm for the complex task of computing the probability of a set of arguments being a complete extension. An experimental evaluation shows promise of our approach.
  arXiv  Detail & Related papers  (2024-07-06T12:08:38Z)
• Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models [62.96551299003463]
  We propose textbftextitThought Propagation (TP) to enhance the complex reasoning ability of Large Language Models. TP first prompts LLMs to propose and solve a set of analogous problems that are related to the input one. TP reuses the results of analogous problems to directly yield a new solution or derive a knowledge-intensive plan for execution to amend the initial solution obtained from scratch.
  arXiv  Detail & Related papers  (2023-10-06T01:40:09Z)
• Rethinking Complex Queries on Knowledge Graphs with Neural Link Predictors [58.340159346749964]
  We propose a new neural-symbolic method to support end-to-end learning using complex queries with provable reasoning capability. We develop a new dataset containing ten new types of queries with features that have never been considered. Our method outperforms previous methods significantly in the new dataset and also surpasses previous methods in the existing dataset at the same time.
  arXiv  Detail & Related papers  (2023-04-14T11:35:35Z)
• ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering [70.6359636116848]
  We propose a new large-scale dataset, ConvFinQA, to study the chain of numerical reasoning in conversational question answering. Our dataset poses great challenge in modeling long-range, complex numerical reasoning paths in real-world conversations.
  arXiv  Detail & Related papers  (2022-10-07T23:48:50Z)
• A Novel DeBERTa-based Model for Financial Question Answering Task [9.083539882647928]
  This research mainly focuses on the financial numerical reasoning dataset - FinQA. In the shared task, the objective is to generate the reasoning program and the final answer according to the given financial report. We obtain an execution accuracy of 68.99 and a program accuracy of 64.53, ranking No. 4 in the 2022 FinQA Challenge.
  arXiv  Detail & Related papers  (2022-07-12T22:34:39Z)
• FinQA: A Dataset of Numerical Reasoning over Financial Data [52.7249610894623]
  We focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. We propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge.
  arXiv  Detail & Related papers  (2021-09-01T00:08:14Z)
• Logically Consistent Loss for Visual Question Answering [66.83963844316561]
  The current advancement in neural-network based Visual Question Answering (VQA) cannot ensure such consistency due to identically distribution (i.i.d.) assumption. We propose a new model-agnostic logic constraint to tackle this issue by formulating a logically consistent loss in the multi-task learning framework. Experiments confirm that the proposed loss formulae and introduction of hybrid-batch leads to more consistency as well as better performance.
  arXiv  Detail & Related papers  (2020-11-19T20:31:05Z)
• Logic-Guided Data Augmentation and Regularization for Consistent Question Answering [55.05667583529711]
  This paper addresses the problem of improving the accuracy and consistency of responses to comparison questions. Our method leverages logical and linguistic knowledge to augment labeled training data and then uses a consistency-based regularizer to train the model.
  arXiv  Detail & Related papers  (2020-04-21T17:03:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.