Task-Level AI Readiness Assessment for Business Process Management:The T-IPO Model and LARA Matrix in Financial-Services IT Operations

• URL: http://arxiv.org/abs/2605.16297v1
• Date: Thu, 16 Apr 2026 10:10:19 GMT
• Title: Task-Level AI Readiness Assessment for Business Process Management:The T-IPO Model and LARA Matrix in Financial-Services IT Operations
• Abstract summary: We describe two design artifacts developed in a financial-services IT setting.<n>LLM Agent Readiness Assessment scores a task's readiness for agent substitution.<n>Pilot deployment of 120 task instances confirms that auto-completion decays monotonically from $95%$ at L1 through about $70%$ at L2 to about $40%$ at L3.
• Score: 1.8200820102702586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Which tasks inside an enterprise workflow can a large-language-model agent reliably handle, and under what conditions? Most business process modeling frameworks still answer this at the activity level, even though a single activity can bundle work of radically different difficulty. This paper takes the analysis a step smaller. We describe two design artifacts developed in a financial-services IT setting: T-IPO, which represents each task as an eight-element tuple, and LARA (LLM Agent Readiness Assessment), a five-dimension rubric that scores a task's readiness for agent substitution. Compliance Sensitivity carries $1.5\times$ weight, a value we fixed through a three-round Delphi study and cross-checked with AHP. The rubric produces four levels, L1 to L4, and applies a floor rule so that a task with maximum compliance load cannot be classified below L3 no matter what the other scores say. Both artifacts sit inside a larger methodology (PARTIS) that we map onto BWW ontology in Section 3. We evaluate the instruments across 127 tasks. Inter-rater agreement reaches Fleiss' $κ= 0.80$; a replication at three further institutions returns $κ= 0.73$. A controlled comparison against activity-level assessment suggests, though does not prove, an improvement in predictive utility at the task level. Pilot deployment of 120 task instances confirms that auto-completion decays monotonically from $95\%$ at L1 through about $70\%$ at L2 to about $40\%$ at L3. Exploratory factor analysis points to a two-factor structure: task readiness seems to be determined jointly by cognitive-execution complexity and governance-compliance intensity. We close with a recalibration procedure (LARA-TCA) so the rubric can keep pace with evolving LLM capabilities.

Related papers

• EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents [8.799203536862382]
  We introduce EcoAgent-Bench, in which every task specifies priced actions and an explicit budget.<n>Our results show that completion under a budget and economical action selection are distinct properties.
  arXiv  Detail & Related papers  (2026-08-06T01:47:47Z)
• SQBench: A Benchmark for Evaluating Task Delivery by Language-Model Agents in Production-Oriented Workflows [0.0]
  We introduce SQBench, a benchmark for evaluating production-oriented task delivery by language-model agents.<n> SQBench v1.0 contains 220 standardized tasks organized into L1 atomic capabilities, L2 composite skills, and L3 business scenarios.<n>Of 2,348 results with Completion = 1, 113 (4.8%) fail the Strict Pass criterion because of risks such as unverifiable citations, inappropriate resource use, or format violations.
  arXiv  Detail & Related papers  (2026-07-25T09:55:31Z)
• A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation [59.98516959731531]
  Abstract reasoning ability reflects intelligence and capacity of LLMs to extract and apply abstract rules.<n>Existing benchmarks either rely on expensive manual annotation, limiting their scale, or risk measuring memorization rather than genuine reasoning.<n>We introduce an automated pipeline named A2RBench, encompassing generation, expansion, evaluation, and analysis.
  arXiv  Detail & Related papers  (2026-05-17T06:14:20Z)
• Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP [3.774094352572544]
  Programmatic state abstraction delivers the largest returns per token spent.<n>Distributing deliberation tools across a hierarchy degrades performance relative to hierarchy alone.<n>Hierarchical decomposition without deliberation achieves the best absolute performance for most models.
  arXiv  Detail & Related papers  (2026-05-15T17:23:08Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• When LLMs Team Up: A Coordinated Attack Framework for Automated Cyber Intrusions [27.62993988829038]
  This paper presents CAESAR, a coordinated multi-agent framework for analysis of intrusion-style tasks.<n>CAESAR decomposes the workflow into five typed roles and coordinates them through a bounded protocol.<n>Compared with a single-agent baseline under matched budgets and tool access, CAESAR improves task success and reduces performance variance.
  arXiv  Detail & Related papers  (2026-05-09T07:47:42Z)
• Agentic Retrieval-Augmented Generation for Financial Document Question Answering [7.56842616602779]
  FinAgent-RAG is an agentic RAG framework that orchestrates iterative retrieval-reasoning loops with self-verification.<n>It integrates a Contrastive Financial Retriever trained with hard negative mining to distinguish semantically similar but numerically distinct financial passages.<n>It achieves 76.81%, 78.46%, and 74.96% execution accuracy respectively, outperforming the strongest baseline by 5.62--9.32 percentage points.
  arXiv  Detail & Related papers  (2026-05-06T19:59:51Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Benchmarking LLM Agents for Wealth-Management Workflows [0.0]
  This dissertation extends TheAgentCompany with a finance-focused environment.<n>It investigates whether a general purpose LLM agent can complete representative wealth-management tasks both accurately and economically.
  arXiv  Detail & Related papers  (2025-12-01T21:56:21Z)
• MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use [27.978435634514128]
  MCPMark is a benchmark designed to evaluate MCP use in a more realistic and comprehensive manner.<n>It consists of $127$ high-quality tasks collaboratively created by domain experts and AI agents.<n>On average, LLMs require $16.2$ execution turns and $17.4$ tool calls per task, significantly surpassing those in previous MCP benchmarks.
  arXiv  Detail & Related papers  (2025-09-28T17:53:27Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• ADaPT: As-Needed Decomposition and Planning with Language Models [131.063805299796]
  We introduce As-Needed Decomposition and Planning for complex Tasks (ADaPT) ADaPT explicitly plans and decomposes complex sub-tasks as-needed, when the Large Language Models is unable to execute them. Our results demonstrate that ADaPT substantially outperforms established strong baselines.
  arXiv  Detail & Related papers  (2023-11-08T17:59:15Z)
• A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration [55.35849138235116]
  We propose automatically selecting a team of agents from candidates to collaborate in a dynamic communication structure toward different tasks and domains. Specifically, we build a framework named Dynamic LLM-Powered Agent Network ($textDyLAN$) for LLM-powered agent collaboration. We demonstrate that DyLAN outperforms strong baselines in code generation, decision-making, general reasoning, and arithmetic reasoning tasks with moderate computational cost.
  arXiv  Detail & Related papers  (2023-10-03T16:05:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.