Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models

• URL: http://arxiv.org/abs/2308.06111v2
• Date: Mon, 14 Aug 2023 07:45:17 GMT
• Title: Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models
• Authors: Lars Hillebrand, Armin Berger, Tobias Deu{\ss}er, Tim Dilmaghani, Mohamed Khaled, Bernd Kliem, R\"udiger Loitz, Maren Pielka, David Leonhard, Christian Bauckhage, Rafet Sifa
• Abstract summary: We present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.
• Score: 2.842163527983814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Auditing financial documents is a very tedious and time-consuming process. As of today, it can already be simplified by employing AI-based solutions to recommend relevant text passages from a report for each legal requirement of rigorous accounting standards. However, these methods need to be fine-tuned regularly, and they require abundant annotated data, which is often lacking in industrial environments. Hence, we present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.

Related papers

• Enhanced Retrieval of Long Documents: Leveraging Fine-Grained Block Representations with Large Language Models [24.02950598944251]
  We introduce a novel, fine-grained approach aimed at enhancing the accuracy of relevance scoring for long documents. Our methodology firstly segments a long document into blocks, each of which is embedded using an LLM. We aggregate the query-block relevance scores through a weighted sum method, yielding a comprehensive score for the query with the entire document.
  arXiv  Detail & Related papers  (2025-01-28T16:03:52Z)
• Boosting Private Domain Understanding of Efficient MLLMs: A Tuning-free, Adaptive, Universal Prompt Optimization Framework [60.26747209785186]
  multimodal large language models (EMLLMs) reduce model size and computational costs and are often deployed on resource-constrained devices. Existing open-sourceLMs rarely have access to private domain-specific data during the pre-training process. We propose a tuntextbfunderlineIng-free, atextbfunderlineDaptivtextbfunderlineE, universtextbfunderlineAL textbfunderlinePrompt Optimization Framework.
  arXiv  Detail & Related papers  (2024-12-27T15:21:17Z)
• Optimizing Multi-Stage Language Models for Effective Text Retrieval [0.0]
  We introduce a novel two-phase text retrieval pipeline optimized for Japanese legal datasets. Our method leverages advanced language models to achieve state-of-the-art performance. To further enhance robustness and adaptability, we incorporate an ensemble model that integrates multiple retrieval strategies.
  arXiv  Detail & Related papers  (2024-12-26T16:05:19Z)
• Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval [6.058427379240698]
  We introduce a new dataset specifically designed for Japanese legal contexts. In the first phase, the model learns a broad understanding of global contexts, enhancing its generalization. In the second phase, the model is fine-tuned to address complex queries specific to legal scenarios. Our pipeline proves effective in English contexts, surpassing comparable baselines on the MS MARCO dataset.
  arXiv  Detail & Related papers  (2024-12-03T10:52:49Z)
• APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching [5.92432068962337]
  Generalized Entity Matching (GEM) aims at judging whether two records represented in different formats refer to the same real-world entity. This paper introduces an augmented prompt tuning framework for the challenges, which consists of two main improvements.
  arXiv  Detail & Related papers  (2024-05-08T05:38:56Z)
• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity [59.57065228857247]
  Retrieval-augmented Large Language Models (LLMs) have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA) We propose a novel adaptive QA framework, that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs based on the query complexity. We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems.
  arXiv  Detail & Related papers  (2024-03-21T13:52:30Z)
• LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking [10.671747198171136]
  We propose a two-stage framework using large language models for ranking-based recommendation (LlamaRec) In particular, we use small-scale sequential recommenders to retrieve candidates based on the user interaction history. LlamaRec consistently achieves datasets superior performance in both recommendation performance and efficiency.
  arXiv  Detail & Related papers  (2023-10-25T06:23:48Z)
• MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation [102.20036684996248]
  We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning. We conduct experiments on two data-to-text generation tasks like WebNLG and LogicNLG.
  arXiv  Detail & Related papers  (2022-12-16T17:36:23Z)
• Classifiers are Better Experts for Controllable Text Generation [63.17266060165098]
  We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. The same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
  arXiv  Detail & Related papers  (2022-05-15T12:58:35Z)
• Text Summarization with Latent Queries [60.468323530248945]
  We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
  arXiv  Detail & Related papers  (2021-05-31T21:14:58Z)
• Conditioned Text Generation with Transfer for Closed-Domain Dialogue Systems [65.48663492703557]
  We show how to optimally train and control the generation of intent-specific sentences using a conditional variational autoencoder. We introduce a new protocol called query transfer that allows to leverage a large unlabelled dataset.
  arXiv  Detail & Related papers  (2020-11-03T14:06:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.