TransactionGPT

• URL: http://arxiv.org/abs/2511.08939v1
• Date: Thu, 13 Nov 2025 01:20:09 GMT
• Title: TransactionGPT
• Abstract summary: TransactionGPT is a foundation model for consumer transaction data within one of world's largest payment networks.<n>We introduce a novel 3D-Transformer architecture specifically tailored for capturing the complex dynamics in payment transaction data.
• Score: 41.91447050171923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present TransactionGPT (TGPT), a foundation model for consumer transaction data within one of world's largest payment networks. TGPT is designed to understand and generate transaction trajectories while simultaneously supporting a variety of downstream prediction and classification tasks. We introduce a novel 3D-Transformer architecture specifically tailored for capturing the complex dynamics in payment transaction data. This architecture incorporates design innovations that enhance modality fusion and computational efficiency, while seamlessly enabling joint optimization with downstream objectives. Trained on billion-scale real-world transactions, TGPT significantly improves downstream classification performance against a competitive production model and exhibits advantages over baselines in generating future transactions. We conduct extensive empirical evaluations utilizing a diverse collection of company transaction datasets spanning multiple downstream tasks, thereby enabling a thorough assessment of TGPT's effectiveness and efficiency in comparison to established methodologies. Furthermore, we examine the incorporation of LLM-derived embeddings within TGPT and benchmark its performance against fine-tuned LLMs, demonstrating that TGPT achieves superior predictive accuracy as well as faster training and inference. We anticipate that the architectural innovations and practical guidelines from this work will advance foundation models for transaction-like data and catalyze future research in this emerging field.

Related papers

• Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings [26.118375969968437]
  Large Language Models (LLMs) can address this limitation through superior semantic understanding.<n>We introduce a hybrid framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models.<n>Our approach employs multi-source data fusion to enrich merchant categorical fields and a one-word constraint principle for consistent embedding generation.
  arXiv  Detail & Related papers  (2025-12-01T23:30:17Z)
• TREASURE: A Transformer-Based Foundation Model for High-Volume Transaction Understanding [33.669519944170816]
  We present TREASURE, a multipurpose transformer-based foundation model specifically designed for transaction data.<n>The model simultaneously captures both consumer behavior and payment network signals, providing comprehensive information necessary for applications like accurate recommendation systems and abnormal behavior detection.<n>We present key insights from extensive ablation studies, benchmarks against production models, and case studies, highlighting valuable knowledge gained from developing TREASURE.
  arXiv  Detail & Related papers  (2025-11-24T20:57:31Z)
• Better with Less: Small Proprietary Models Surpass Large Language Models in Financial Transaction Understanding [1.4125114383423856]
  This paper conducts experiments to evaluate three types of Transformer models: pretrained LLMs, fine-tuned LLMs, and small proprietary models developed from scratch.<n>Our findings highlight the importance of model selection based on domain-specific needs.
  arXiv  Detail & Related papers  (2025-09-30T05:23:08Z)
• The carbon cost of materials discovery: Can machine learning really accelerate the discovery of new photovoltaics? [0.05524804393257919]
  Computational screening has become a powerful complement to experimental efforts in the discovery of high-performance photovoltaic (PV) materials.<n>Most rely on density functional theory (DFT) to estimate electronic and optical properties relevant to solar energy conversion.<n>Machine learning (ML) models have recently gained attention as surrogates for DFT, offering drastic reductions in resource use with competitive predictive performance.
  arXiv  Detail & Related papers  (2025-07-17T15:55:02Z)
• BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline [34.518474035662905]
  General capabilities of Large Language Models (LLM) highly rely on extensive pretraining datasets, treated as commercial secrets by several institutions. We open-source the details of a universally applicable data processing pipeline to validate its effectiveness and potential. BaichuanSEED demonstrates consistency and predictability throughout training and achieves comparable performance on comprehensive benchmarks.
  arXiv  Detail & Related papers  (2024-08-27T14:08:23Z)
• Implicitly Guided Design with PropEn: Match your Data to Follow the Gradient [52.2669490431145]
  PropEn is inspired by'matching', which enables implicit guidance without training a discriminator. We show that training with a matched dataset approximates the gradient of the property of interest while remaining within the data distribution.
  arXiv  Detail & Related papers  (2024-05-28T11:30:19Z)
• A Bargaining-based Approach for Feature Trading in Vertical Federated Learning [54.51890573369637]
  We propose a bargaining-based feature trading approach in Vertical Federated Learning (VFL) to encourage economically efficient transactions. Our model incorporates performance gain-based pricing, taking into account the revenue-based optimization objectives of both parties.
  arXiv  Detail & Related papers  (2024-02-23T10:21:07Z)
• Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark [95.0484665958967]
  Pre-trained vision models (PVMs) have demonstrated remarkable adaptability across a wide range of downstream vision tasks.<n>As these models scale to billions or even trillions of parameters, conventional full fine-tuning has become increasingly impractical due to its high computational and storage demands.<n> parameter-efficient fine-tuning (PEFT) has emerged as a promising alternative, aiming to achieve performance comparable to full fine-tuning while making minimal adjustments to the model parameters.
  arXiv  Detail & Related papers  (2024-02-03T19:12:20Z)
• A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking [78.2700757742992]
  Task-oriented dialogue systems often employ a Dialogue State Tracker (DST) to successfully complete conversations. Recent state-of-the-art DST implementations rely on schemata of diverse services to improve model robustness. We propose a single multi-task BERT-based model that jointly solves the three DST tasks of intent prediction, requested slot prediction and slot filling.
  arXiv  Detail & Related papers  (2022-07-02T13:27:59Z)
• Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval [152.3504607706575]
  This research aims to conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories. We first contribute the Product1M datasets, and define two real practical instance-level retrieval tasks. We exploit to train a more effective cross-modal model which is adaptively capable of incorporating key concept information from the multi-modal data.
  arXiv  Detail & Related papers  (2022-06-17T15:40:45Z)
• Data transformation based optimized customer churn prediction model for the telecommunication industry [1.3100459580864314]
  This study is conducted in a customer churn prediction (CCP) context in the telecommunication industry. We have proposed a novel approach of combining data transformation methods with the machine learning models for the CCP problem.
  arXiv  Detail & Related papers  (2022-01-11T17:23:55Z)
• Portfolio Optimization with 2D Relative-Attentional Gated Transformer [9.541129630971689]
  We propose a novel Deterministic Policy Gradient with 2D Relative-attentional Gated Transformer (DPGRGT) model. Applying learnable relative positional embeddings for the time and assets axes, the model better understands the peculiar structure of the financial data. In our experiment using U.S. stock market data of 20 years, our model outperformed baseline models and demonstrated its effectiveness.
  arXiv  Detail & Related papers  (2020-12-27T14:08:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.