FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models

• URL: http://arxiv.org/abs/2406.01618v1
• Date: Tue, 28 May 2024 16:34:24 GMT
• Title: FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models
• Authors: Anjanava Biswas, Wrick Talukdar,
• Abstract summary: FinEmbedDiff is a cost-effective vector sampling method to classify financial documents. It achieves competitive classification accuracy compared to state-of-the-art baselines. It is a practical and scalable solution for real-world financial applications.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurate classification of multi-modal financial documents, containing text, tables, charts, and images, is crucial but challenging. Traditional text-based approaches often fail to capture the complex multi-modal nature of these documents. We propose FinEmbedDiff, a cost-effective vector sampling method that leverages pre-trained multi-modal embedding models to classify financial documents. Our approach generates multi-modal embedding vectors for documents, and compares new documents with pre-computed class embeddings using vector similarity measures. Evaluated on a large dataset, FinEmbedDiff achieves competitive classification accuracy compared to state-of-the-art baselines while significantly reducing computational costs. The method exhibits strong generalization capabilities, making it a practical and scalable solution for real-world financial applications.

Related papers

• Multi-modal Retrieval Augmented Multi-modal Generation: Datasets, Evaluation Metrics and Strong Baselines [64.61315565501681]
  Multi-modal Retrieval Augmented Multi-modal Generation (M$2$RAG) is a novel task that enables foundation models to process multi-modal web content. Despite its potential impact, M$2$RAG remains understudied, lacking comprehensive analysis and high-quality data resources.
  arXiv  Detail & Related papers  (2024-11-25T13:20:19Z)
• Multimodality Helps Few-Shot 3D Point Cloud Semantic Segmentation [61.91492500828508]
  Few-shot 3D point cloud segmentation (FS-PCS) aims at generalizing models to segment novel categories with minimal support samples. We introduce a cost-free multimodal FS-PCS setup, utilizing textual labels and the potentially available 2D image modality. We propose a simple yet effective Test-time Adaptive Cross-modal Seg (TACC) technique to mitigate training bias.
  arXiv  Detail & Related papers  (2024-10-29T19:28:41Z)
• Out-of-Distribution Detection with Attention Head Masking for Multimodal Document Classification [3.141006099594433]
  We propose a novel methodology termed as attention head masking (AHM) for multi-modal OOD tasks in document classification systems. Our empirical results demonstrate that the proposed AHM method outperforms all state-of-the-art approaches. To address the scarcity of high-quality publicly available document datasets, we introduce FinanceDocs, a new document AI dataset.
  arXiv  Detail & Related papers  (2024-08-20T23:30:00Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences [0.0]
  We present a generative pretraining method that can be used to obtain contextualised embeddings of financial transactions. We additionally perform large-scale pretraining of an embedding model using a corpus of data from 180 issuing banks containing 5.1 billion transactions.
  arXiv  Detail & Related papers  (2024-01-03T09:32:48Z)
• mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model [73.38800189095173]
  This work focuses on strengthening the multi-modal diagram analysis ability of Multimodal LLMs. By parsing Latex source files of high-quality papers, we carefully build a multi-modal diagram understanding dataset M-Paper. M-Paper is the first dataset to support joint comprehension of multiple scientific diagrams, including figures and tables in the format of images or Latex codes.
  arXiv  Detail & Related papers  (2023-11-30T04:43:26Z)
• UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding [93.92313947913831]
  We introduce UniDoc, a novel multimodal model equipped with text detection and recognition capabilities. To the best of our knowledge, this is the first large multimodal model capable of simultaneous text detection, recognition, spotting, and understanding.
  arXiv  Detail & Related papers  (2023-08-19T17:32:34Z)
• $\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation [65.29170569821093]
  parallel text generation has received widespread attention due to its success in generation efficiency. In this paper, we propose $textitlatent$-GLAT, which employs the discrete latent variables to capture word categorical information. Experiment results show that our method outperforms strong baselines without the help of an autoregressive model.
  arXiv  Detail & Related papers  (2022-04-05T07:34:12Z)
• Efficient Classification of Long Documents Using Transformers [13.927622630633344]
  We evaluate the relative efficacy measured against various baselines and diverse datasets. Results show that more complex models often fail to outperform simple baselines and yield inconsistent performance across datasets.
  arXiv  Detail & Related papers  (2022-03-21T18:36:18Z)
• Sparse Fusion for Multimodal Transformers [7.98117428941095]
  We present Sparse Fusion Transformers (SFT), a novel multimodal fusion method for transformers. Key to our idea is a sparse-pooling block that reduces unimodal token sets prior to cross-modality modeling. State-of-the-art performance is obtained on multiple benchmarks under similar experiment conditions, while reporting up to six-fold reduction in computational cost and memory requirements.
  arXiv  Detail & Related papers  (2021-11-23T16:43:49Z)
• Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity [11.157086694203201]
  We present a new scientific document similarity model based on matching fine-grained aspects. Our model is trained using co-citation contexts that describe related paper aspects as a novel form of textual supervision.
  arXiv  Detail & Related papers  (2021-11-16T11:12:30Z)
• Single-Modal Entropy based Active Learning for Visual Question Answering [75.1682163844354]
  We address Active Learning in the multi-modal setting of Visual Question Answering (VQA) In light of the multi-modal inputs, image and question, we propose a novel method for effective sample acquisition. Our novel idea is simple to implement, cost-efficient, and readily adaptable to other multi-modal tasks.
  arXiv  Detail & Related papers  (2021-10-21T05:38:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.