Better with Less: Small Proprietary Models Surpass Large Language Models in Financial Transaction Understanding

• URL: http://arxiv.org/abs/2509.25803v1
• Date: Tue, 30 Sep 2025 05:23:08 GMT
• Title: Better with Less: Small Proprietary Models Surpass Large Language Models in Financial Transaction Understanding
• Authors: Wanying Ding, Savinay Narendra, Xiran Shi, Adwait Ratnaparkhi, Chengrui Yang, Nikoo Sabzevar, Ziyan Yin,
• Abstract summary: This paper conducts experiments to evaluate three types of Transformer models: pretrained LLMs, fine-tuned LLMs, and small proprietary models developed from scratch.<n>Our findings highlight the importance of model selection based on domain-specific needs.
• Score: 1.4125114383423856
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Analyzing financial transactions is crucial for ensuring regulatory compliance, detecting fraud, and supporting decisions. The complexity of financial transaction data necessitates advanced techniques to extract meaningful insights and ensure accurate analysis. Since Transformer-based models have shown outstanding performance across multiple domains, this paper seeks to explore their potential in understanding financial transactions. This paper conducts extensive experiments to evaluate three types of Transformer models: Encoder-Only, Decoder-Only, and Encoder-Decoder models. For each type, we explore three options: pretrained LLMs, fine-tuned LLMs, and small proprietary models developed from scratch. Our analysis reveals that while LLMs, such as LLaMA3-8b, Flan-T5, and SBERT, demonstrate impressive capabilities in various natural language processing tasks, they do not significantly outperform small proprietary models in the specific context of financial transaction understanding. This phenomenon is particularly evident in terms of speed and cost efficiency. Proprietary models, tailored to the unique requirements of transaction data, exhibit faster processing times and lower operational costs, making them more suitable for real-time applications in the financial sector. Our findings highlight the importance of model selection based on domain-specific needs and underscore the potential advantages of customized proprietary models over general-purpose LLMs in specialized applications. Ultimately, we chose to implement a proprietary decoder-only model to handle the complex transactions that we previously couldn't manage. This model can help us to improve 14% transaction coverage, and save more than \$13 million annual cost.

Related papers

• Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings [26.118375969968437]
  Large Language Models (LLMs) can address this limitation through superior semantic understanding.<n>We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models.<n>Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation.
  arXiv  Detail & Related papers  (2025-12-01T23:30:17Z)
• Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning [20.41220110321494]
  We propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning.<n>STEER is a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger language models.<n>Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing.
  arXiv  Detail & Related papers  (2025-11-09T02:33:08Z)
• Think Then Embed: Generative Context Improves Multimodal Embedding [51.76690812535934]
  We propose a Think-Then-Embed (TTE) framework for Universal Multimodal Embeddings (UME), composed of a reasoner and an embedder.<n>By leveraging a powerful MLLM reasoner, we achieve state-of-the-art performance on the MMEB-V2 benchmark, surpassing proprietary models trained on massive in-house datasets.
  arXiv  Detail & Related papers  (2025-10-06T16:53:56Z)
• Your Spending Needs Attention: Modeling Financial Habits with Transformers [2.5960274245156922]
  This paper investigates using transformer-based representation learning models for transaction data.<n>We propose a new method enabling the use of SSL with transaction data by adapting transformer-based models to handle both textual and structured attributes.
  arXiv  Detail & Related papers  (2025-07-31T05:56:21Z)
• Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches [3.549427092296418]
  Large language models (LLMs) offer new opportunities for interacting with complex software artifacts, such as software models, through natural language.<n>This paper investigates two approaches for leveraging LLMs to answer questions over software models.<n>We evaluate these approaches using an Ecore metamodel designed for timing analysis and software optimization in automotive domains.
  arXiv  Detail & Related papers  (2025-06-16T07:34:28Z)
• Learnware of Language Models: Specialized Small Language Models Can Do Big [50.285859986475394]
  This paper presents a preliminary attempt to apply the learnware paradigm to language models.<n>We simulated a learnware system comprising approximately 100 learnwares of specialized SLMs with 8B parameters.<n>By selecting one suitable learnware for each task-specific inference, the system outperforms the base SLMs on all benchmarks.
  arXiv  Detail & Related papers  (2025-05-19T17:54:35Z)
• STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Trading [55.02735046724146]
  In financial trading, factor models are widely used to price assets and capture excess returns from mispricing.<n>We propose a Spatio-Temporal factOR Model based on dual vector quantized variational autoencoders, named STORM.<n>Storm extracts features of stocks from temporal and spatial perspectives, then fuses and aligns these features at the fine-grained and semantic level, and represents the factors as multi-dimensional embeddings.
  arXiv  Detail & Related papers  (2024-12-12T17:15:49Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• SNFinLLM: Systematic and Nuanced Financial Domain Adaptation of Chinese Large Language Models [6.639972934967109]
  Large language models (LLMs) have become powerful tools for advancing natural language processing applications in the financial industry. We propose a novel large language model specifically designed for the Chinese financial domain, named SNFinLLM. SNFinLLM excels in domain-specific tasks such as answering questions, summarizing financial research reports, analyzing sentiment, and executing financial calculations.
  arXiv  Detail & Related papers  (2024-08-05T08:24:24Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Large Language Model Adaptation for Financial Sentiment Analysis [2.0499240875882]
  Generalist language models tend to fall short in tasks specifically tailored for finance. Two foundation models with less than 1.5B parameters have been adapted using a wide range of strategies. We show that small LLMs have comparable performance to larger scale models, while being more efficient in terms of parameters and data.
  arXiv  Detail & Related papers  (2024-01-26T11:04:01Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.