MINT: A Universal Zero-Shot Predictor for Transaction Data

• URL: http://arxiv.org/abs/2608.14198v1
• Date: Fri, 14 Aug 2026 11:17:01 GMT
• Title: MINT: A Universal Zero-Shot Predictor for Transaction Data
• Abstract summary: We present MINT, a framework that connects a pretrained transaction sequence encoder to a decoder-only LLM through lightweight embedding injection.<n>MINT achieves state-of-the-art predictive question-answering performance in both in-distribution and out-of-distribution questions.<n>We find that MINT achieves state-of-the-art predictive question-answering performance in both in-distribution and out-of-distribution questions.
• Score: 5.76828154035244
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Banks analyse sequential financial transaction data to perform many tasks, including fraud prevention, credit risk assessment and offer personalization. To improve the predictive accuracy of these tasks, Payments Foundation Models encode transaction sequence data as rich contextual embeddings, which can then be provided to task-specific models as features. However, these Foundation Models are not designed for flexible zero-shot reasoning across novel downstream prediction tasks, limiting their adaptability and utility. Existing LLM-based approaches to zero-shot prediction often fail to fully exploit the predictive signal within transaction data, while relying on costly text serialization or task-specific architectures that scale poorly. To address these limitations, we present the Multimodal Instruction Network for Transactions (MINT), a framework that connects a pretrained transaction sequence encoder to a decoder-only LLM through lightweight embedding injection, transaction-language alignment, and instruction tuning. We find that MINT achieves state-of-the-art predictive question-answering performance in both in-distribution and out-of-distribution questions, while substantially reducing input tokens, latency, and memory consumption compared to text-serialization baselines. Through comprehensive analyses of representations, alignment strategies, training data, and history length, we establish that compact transaction embeddings are a superior approach to transaction representation than text serialization for multimodal reasoning and zero-shot prediction tasks.

Related papers

• Financial Transaction Retrieval and Contextual Evidence for Knowledge-Grounded Reasoning [0.3947144942254943]
  FinTRACE is a retrieval-first architecture that converts raw transactions into reusable feature representations.<n>Across public and industrial benchmarks, FinTRACE substantially improves low-supervision transaction analytics.
  arXiv  Detail & Related papers  (2026-03-16T15:56:21Z)
• Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequences [5.03287845497428]
  Industrial financial systems operate on temporal event sequences such as transactions, user actions, and system logs.<n>We introduce Embedding-Aware Feature Discovery (EAFD), a unified framework that bridges the gap between learned embeddings and feature-based pipelines.<n>EAFD iteratively discovers, evaluates, and refines features directly from raw event sequences using two complementary criteria.
  arXiv  Detail & Related papers  (2026-03-16T14:29:26Z)
• Fine-tuning MLLMs Without Forgetting Is Easier Than You Think [72.59321247529975]
  We design a 2x2 experimental framework to assess model performance across in-distribution and out-of-distribution image and text inputs.<n>Our results show that appropriate regularization, such as constraining the number of trainable parameters or adopting a low learning rate, effectively prevents forgetting when dealing with out-of-distribution images.<n>We attribute this forgetting as task-specific overfitting and address this issue by introducing a data-hybrid training strategy.
  arXiv  Detail & Related papers  (2026-03-15T17:16:19Z)
• Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs [66.63911043019294]
  Data preparation aims to denoise raw datasets, uncover cross-dataset relationships, and extract valuable insights from them.<n>This paper focuses on the use of LLM techniques to prepare data for diverse downstream tasks.<n>We introduce a task-centric taxonomy that organizes the field into three major tasks: data cleaning, standardization, error processing, imputation, data integration, and data enrichment.
  arXiv  Detail & Related papers  (2026-01-22T12:02:45Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings [26.118375969968437]
  Large Language Models (LLMs) can address this limitation through superior semantic understanding.<n>We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models.<n>Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation.
  arXiv  Detail & Related papers  (2025-12-01T23:30:17Z)
• LOBERT: Generative AI Foundation Model for Limit Order Book Messages [52.54676013594764]
  We introduce LOBERT, a general-purpose encoder-only foundation model for financial Limit Order Books (LOB) data.<n>LOBERT adapts the original BERT architecture for LOB data by using a novel tokenization scheme that treats complete multi-dimensional messages as single tokens.<n>With these methods, LOBERT achieves leading performance in tasks such as predicting mid-price movements and next messages.
  arXiv  Detail & Related papers  (2025-11-16T11:42:48Z)
• Semi-Supervised Bayesian GANs with Log-Signatures for Uncertainty-Aware Credit Card Fraud Detection [0.0]
  We present a novel deep generative semi-supervised framework for credit card fraud detection, formulated as time series classification task.<n>We extend conditional Geneversarative Adrial Networks (GANs) for targeted data augmentation, integrate Bayesian inference to obtain predictive distributions and quantify uncertainty.<n>Our approach is evaluated on the BankSim dataset, a widely used simulator for credit card transaction data, under varying proportions of labeled samples.
  arXiv  Detail & Related papers  (2025-08-31T16:57:02Z)
• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks.<n>We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Discrete Key-Value Bottleneck [95.61236311369821]
  Deep neural networks perform well on classification tasks where data streams are i.i.d. and labeled data is abundant. One powerful approach that has addressed this challenge involves pre-training of large encoders on volumes of readily available data, followed by task-specific tuning. Given a new task, however, updating the weights of these encoders is challenging as a large number of weights needs to be fine-tuned, and as a result, they forget information about the previous tasks. We propose a model architecture to address this issue, building upon a discrete bottleneck containing pairs of separate and learnable key-value codes.
  arXiv  Detail & Related papers  (2022-07-22T17:52:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.