Rethinking LLMOps for Fraud and AML: Building a Compliance-Grade LLM Serving Stack

• URL: http://arxiv.org/abs/2605.11232v1
• Date: Mon, 11 May 2026 20:47:41 GMT
• Title: Rethinking LLMOps for Fraud and AML: Building a Compliance-Grade LLM Serving Stack
• Abstract summary: Fraud and anti-money-laundering (AML) compliance are high-value domains for large language models (LLMs)<n>Compliance prompts are often prefix-heavy, schema-constrained, and evidence-rich.<n>These properties make prefix reuse, KV-cache efficiency, runtime tuning, model orchestration, and output validation first-order systems concerns.
• Score: 2.299706695317804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fraud detection and anti-money-laundering (AML) compliance are high-value domains for large language models (LLMs), but their serving requirements differ sharply from generic chat workloads. Compliance prompts are often prefix-heavy, schema-constrained, and evidence-rich, combining reusable policy instructions, risk taxonomies, transaction or document context, and short structured outputs such as JSON labels or risk factors. These properties make prefix reuse, KV-cache efficiency, runtime tuning, model orchestration, and output validation first-order systems concerns. This paper introduces a workload-aware LLMOps stack for fraud and AML workloads using self-hosted open-weight models such as Meta Llama and Alibaba Qwen. The stack combines vLLM-style runtime tuning, PagedAttention, Automatic Prefix Caching, multi-adapter serving, adapter and prompt-length-aware batching, sleep/wake lifecycle management, speculative decoding, and optional prefill/decode disaggregation. To avoid exposing institution-specific data, the reproducibility track converts public synthetic AML datasets, including IBM AML and SAML-D, into prefix-heavy compliance prompts with reusable policy text, transaction evidence, typology definitions, and schema-constrained outputs. We also incorporate an LLM-as-judge quality gate using deterministic compliance checks, reference metrics, expert-adjudicated calibration data where available, and multi-judge rubric scoring. Across public-synthetic AML workloads and controlled serving benchmarks, workload-aware tuning improved throughput from 612-650 to 3,600 requests/hour, reduced P99 latency from 31-38 seconds to 6.4-8.7 seconds, and increased GPU utilization from 12% to 78%. These results show that regulated LLM performance is a workload-design, serving-optimization, and quality-gating problem, not only a model-selection problem.

Related papers

• Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner [55.80248281761638]
  multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning.<n>We propose DG-Mem, a dual-grained agentic memory framework that augments a frozen MLLM with a non- externally stored memory built once from training-time rollouts and consulted read-only at test time.
  arXiv  Detail & Related papers  (2026-08-24T13:56:01Z)
• Making Failure Safe: A Constrained, Verifiable Agent Framework for Open-Web Data Collection [4.093009863847175]
  LLMs and agents can generate web scrapers from natural-language requirements, but direct generation remains unreliable.<n>We propose a constrained, verifiable agent framework that shifts LLM output from free-form code to typed collector configurations.
  arXiv  Detail & Related papers  (2026-06-25T14:05:37Z)
• PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalization [49.659739762240655]
  Generative listwise ranking with Large Multimodal Models (LMMs) aims to capture global list context in a single forward pass, but its effectiveness degrades in long-context multimodal scenarios.<n>We identify a recurring failure mode, parse collapse, where the autoregressive decoder produces fluent yet incomplete rankings by silently omitting candidates and terminating early.<n>We propose PRISMR, a framework that replaces transient in-context list processing with parametric structural conditioning.
  arXiv  Detail & Related papers  (2026-06-11T06:09:51Z)
• Think Fast, Talk Smart: Partitioning Deterministic and Neural Computation for Structured Health Text Generation [0.0764671395172401]
  Large language models (LLMs) are increasingly being used to generate health text from structured records such as wearable time series, biomarkers, vitals, and care-management logs.<n>We ask which responsibilities in structured health generation should be deterministic runtime rather than LLM prompting.<n>We introduce Think Fast, Talk Smart, a sleep-health insight pipeline in which deterministic code performs recurring analysis before one bounded LLM writer call.
  arXiv  Detail & Related papers  (2026-05-28T09:16:21Z)
• An Interpretable Latency Model for Speculative Decoding in LLM Serving [39.74508663765912]
  We develop a simple and interpretable latency model for Speculative decoding in large language model serving systems.<n>We validate our model using extensive measurements from vLLM across verifier and drafter model sizes, prefill and decode lengths, request rates, draft lengths, and acceptance probabilities.<n>We show how the framework extends to mixture of experts models, where sparse expert activation changes the effective service costs across load regimes.
  arXiv  Detail & Related papers  (2026-05-14T16:45:28Z)
• Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement [8.059802912761919]
  We uncover a systematic failure of large language models (LLMs) in matching code to natural language requirements.<n>More detailed prompt design, particularly with those requiring explanations and proposed corrections, leads to higher misjudgment rates.<n>We propose a Fix-guided Verification Filter that treats the model proposed fix as executable counterfactual evidence.
  arXiv  Detail & Related papers  (2026-02-28T08:35:25Z)
• DiffuRank: Effective Document Reranking with Diffusion Language Models [71.16830004674513]
  We propose DiffuRank, a reranking framework built upon diffusion language models (dLLMs)<n>dLLMs support more flexible decoding and generation processes that are not constrained to a left-to-right order.<n>We show dLLMs achieve performance comparable to, and in some cases exceeding, that of autoregressive LLMs with similar model sizes.
  arXiv  Detail & Related papers  (2026-02-13T02:18:14Z)
• SafeLoad: Efficient Admission Control Framework for Identifying Memory-Overloading Queries in Cloud Data Warehouses [59.68732483257323]
  Memory overload is a common form of resource exhaustion in cloud data warehouses.<n>We propose SafeLoad, the first query admission control framework specifically designed to identify memory-overloading (MO) queries.<n>We show that SafeLoad achieves state-of-the-art prediction performance with low online and offline time overhead.
  arXiv  Detail & Related papers  (2026-01-05T08:29:51Z)
• Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs [0.10742675209112619]
  Small language models (SLMs; 1-12B params, sometimes up to 20B) are sufficient and often superior for agentic workloads.<n>We synthesize recent evidence across open and proprietary SLMs and connect it to modern evaluations.<n>We formalize SLM-fallback systems with uncertainty-aware routing and verifier cascades, and propose engineering metrics that reflect real production goals.
  arXiv  Detail & Related papers  (2025-10-04T15:48:04Z)
• SweRank: Software Issue Localization with Code Ranking [109.3289316191729]
  SweRank is an efficient retrieve-and-rerank framework for software issue localization.<n>We construct SweLoc, a large-scale dataset curated from public GitHub repositories.<n>We show that SweRank achieves state-of-the-art performance, outperforming both prior ranking models and costly agent-based systems.
  arXiv  Detail & Related papers  (2025-05-07T19:44:09Z)
• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.<n>LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.<n>Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• Open-domain Implicit Format Control for Large Language Model Generation [52.83173553689678]
  We introduce a novel framework for controlled generation in large language models (LLMs) This study investigates LLMs' capabilities to follow open-domain, one-shot constraints and replicate the format of the example answers. We also develop a dataset collection methodology for supervised fine-tuning that enhances the open-domain format control of LLMs without degrading output quality.
  arXiv  Detail & Related papers  (2024-08-08T11:51:45Z)
• TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput [37.56866491624234]
  Large Language Model (LLM) serving systems batch concurrent user requests to achieve efficient serving.<n>We present TurboSpec, a speculation control system that automatically profiles the execution environment.<n>We demonstrate its effectiveness across diverse workloads and hardware configurations.
  arXiv  Detail & Related papers  (2024-06-20T07:43:33Z)
• Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction [8.705908108054878]
  Large models (LLMs) have been driving a new wave of AI applications across numerous domains. We present a speculative shortest-job-first (SSJF) scheduler that uses a light proxy model to predict LLM output sequence lengths.
  arXiv  Detail & Related papers  (2024-04-12T14:46:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.