Synthetic Data-Driven Prompt Tuning for Financial QA over Tables and Documents

• URL: http://arxiv.org/abs/2511.06292v2
• Date: Fri, 14 Nov 2025 05:38:49 GMT
• Title: Synthetic Data-Driven Prompt Tuning for Financial QA over Tables and Documents
• Authors: Yaoning Yu, Kai-Min Chang, Ye Yu, Kai Wei, Haojing Luo, Haohan Wang,
• Abstract summary: We introduce a self-improving prompt framework driven by data-augmented optimization.<n>We generate synthetic financial tables and document excerpts, verify their correctness and robustness then update the prompt based on the results.<n>We achieve higher performance in both accuracy and robustness than standard prompt methods.
• Score: 21.737958911422805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Financial documents like earning reports or balance sheets often involve long tables and multi-page reports. Large language models have become a new tool to help numerical reasoning and understanding these documents. However, prompt quality can have a major effect on how well LLMs perform these financial reasoning tasks. Most current methods tune prompts on fixed datasets of financial text or tabular data, which limits their ability to adapt to new question types or document structures, or they involve costly and manually labeled/curated dataset to help build the prompts. We introduce a self-improving prompt framework driven by data-augmented optimization. In this closed-loop process, we generate synthetic financial tables and document excerpts, verify their correctness and robustness, and then update the prompt based on the results. Specifically, our framework combines a synthetic data generator with verifiers and a prompt optimizer, where the generator produces new examples that exposes weaknesses in the current prompt, the verifiers check the validity and robustness of the produced examples, and the optimizer incrementally refines the prompt in response. By iterating these steps in a feedback cycle, our method steadily improves prompt accuracy on financial reasoning tasks without needing external labels. Evaluation on DocMath-Eval benchmark demonstrates that our system achieves higher performance in both accuracy and robustness than standard prompt methods, underscoring the value of incorporating synthetic data generation into prompt learning for financial applications.

Related papers

• Structure First, Reason Next: Enhancing a Large Language Model using Knowledge Graph for Numerical Reasoning in Financial Documents [0.21485350418225244]
  Large Language Models (LLMs) have shown promising results in multiple Question-Answering (Q-A) systems.<n>Structured data augmentations, such as Knowledge Graphs (KGs), have notably improved the predictions of LLMs.<n>This paper proposes a framework to incorporate structured information using KGs along with LLM predictions for numerical reasoning tasks.
  arXiv  Detail & Related papers  (2026-01-12T17:39:08Z)
• Enhancing Business Analytics through Hybrid Summarization of Financial Reports [0.152292571922932]
  Financial reports and earnings communications contain large volumes of structured and semi structured information.<n>We present a hybrid summarization framework that combines extractive and abstractive techniques to produce concise and factually reliable summaries.<n>These findings support the development of practical summarization systems for distilling lengthy financial texts into usable business insights.
  arXiv  Detail & Related papers  (2025-12-28T16:25:12Z)
• Metadata-Driven Retrieval-Augmented Generation for Financial Question Answering [0.0]
  We introduce a sophisticated indexing pipeline to create contextually rich document chunks.<n>We benchmark a spectrum of enhancements, including pre-retrieval filtering, post-retrieval reranking, and enriched embeddings.<n>Our proposed optimal architecture combines LLM-driven pre-retrieval optimizations with these contextual embeddings to achieve superior performance.
  arXiv  Detail & Related papers  (2025-10-28T13:16:36Z)
• FinSight: Towards Real-World Financial Deep Research [68.31086471310773]
  FinSight is a novel framework for producing high-quality, multimodal financial reports.<n>To ensure professional-grade visualization, we propose an Iterative Vision-Enhanced Mechanism.<n>A two-stage Writing Framework expands concise Chain-of-Analysis segments into coherent, citation-aware, and multimodal reports.
  arXiv  Detail & Related papers  (2025-10-19T14:05:35Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [57.43420753842626]
  FinLFQA is a benchmark designed to evaluate the ability of Large Language Models to generate long-form answers to complex financial questions.<n>We provide an automatic evaluation framework covering both answer quality and attribution quality.
  arXiv  Detail & Related papers  (2025-10-07T20:06:15Z)
• LLM-Symbolic Integration for Robust Temporal Tabular Reasoning [69.27153114778748]
  We introduce TempTabQA-C, a synthetic dataset designed for systematic and controlled evaluations.<n>This structured approach allows Large Language Models (LLMs) to generate and executesql queries, enhancing generalization and mitigating biases.
  arXiv  Detail & Related papers  (2025-06-06T05:14:04Z)
• Structuring the Unstructured: A Multi-Agent System for Extracting and Querying Financial KPIs and Guidance [54.25184684077833]
  We propose an efficient and scalable method for extracting quantitative insights from unstructured financial documents.<n>Our proposed system consists of two specialized agents: the emphExtraction Agent and the emphText-to-Agent
  arXiv  Detail & Related papers  (2025-05-25T15:45:46Z)
• An Efficient Deep Learning-Based Approach to Automating Invoice Document Validation [0.0]
  We propose to automate the validation of machine written invoices using document layout analysis and object detection techniques.<n>We introduce a novel dataset consisting of manually annotated real-world invoices and a multi-criteria validation process.
  arXiv  Detail & Related papers  (2025-03-15T21:33:00Z)
• Quantifying Qualitative Insights: Leveraging LLMs to Market Predict [0.0]
  This study addresses challenges by leveraging daily reports from securities firms to create high-quality contextual information. The reports are segmented into text-based key factors and combined with numerical data, such as price information, to form context sets. A crafted prompt is designed to assign scores to the key factors, converting qualitative insights into quantitative results.
  arXiv  Detail & Related papers  (2024-11-13T07:45:40Z)
• SEC-QA: A Systematic Evaluation Corpus for Financial QA [12.279234447220155]
  Existing datasets are often constrained by size, context, or relevance to practical applications. We propose SEC-QA, a continuous dataset generation framework with two key features. We introduce a QA system based on program-of-thought that improves the ability to perform complex information retrieval and quantitative reasoning pipelines.
  arXiv  Detail & Related papers  (2024-06-20T15:12:41Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.