Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

• URL: http://arxiv.org/abs/2603.10807v1
• Date: Wed, 11 Mar 2026 14:14:13 GMT
• Title: Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
• Abstract summary: We propose a risk-aware evaluation framework for security failures in Banking, Financial Services, and Insurance.<n>We combine a domain-specific taxonomy of financial harms, an automated multi-round red-teaming pipeline, and an ensemble-based judging protocol.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid adoption of large language models (LLMs) in financial services introduces new operational, regulatory, and security risks. Yet most red-teaming benchmarks remain domain-agnostic and fail to capture failure modes specific to regulated BFSI settings, where harmful behavior can be elicited through legally or professionally plausible framing. We propose a risk-aware evaluation framework for LLM security failures in Banking, Financial Services, and Insurance (BFSI), combining a domain-specific taxonomy of financial harms, an automated multi-round red-teaming pipeline, and an ensemble-based judging protocol. We introduce the Risk-Adjusted Harm Score (RAHS), a risk-sensitive metric that goes beyond success rates by quantifying the operational severity of disclosures, accounting for mitigation signals, and leveraging inter-judge agreement. Across diverse models, we find that higher decoding stochasticity and sustained adaptive interaction not only increase jailbreak success, but also drive systematic escalation toward more severe and operationally actionable financial disclosures. These results expose limitations of single-turn, domain-agnostic security evaluation and motivate risk-sensitive assessment under prolonged adversarial pressure for real-world BFSI deployment.

Related papers

• Safe Remediation as Risk-Constrained Intervention Decision in Microservice Systems [2.1686718805804976]
  In modern IT operations (IT-Ops), the cost of an incorrect repair often exceeds the cost of no action at all.<n>This paper makes three contributions to close this gap: (i) we reformulate safe remediation as a risk-constrained intervention decision problem; (ii) we introduce a three-dimensional risk decomposition; and (iii) we design a context-adaptive human-in-the-loop (HITL) gate that turns from a binary failsafe into a bandwidth-aware control layer.
  arXiv  Detail & Related papers  (2026-07-22T10:43:14Z)
• Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents [0.0]
  Risk-Aware Causal Gating (RACG) is a framework that decides whether to act on, defer, or abstain from a model's prediction.<n>RACG models the causal pathway from candidate actions to outcomes and gates each decision according to an estimated counterfactual risk.<n>We show that RACG reduces high-cost errors substantially while preserving most of the utility of an ungated policy.
  arXiv  Detail & Related papers  (2026-06-11T20:19:35Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• Towards a Risk-Cost Model for Financial Adaptive Authentication [9.133320151595084]
  We introduce a formal Risk-Cost Model (RCM) for adaptive authentication in financial systems.<n>The RCM integrates three essential components: (i) cost-sensitive risk functions that explicitly capture fraud loss, opportunity cost, and tail risk through Conditional Value-at-Risk (CVaR), and (ii) sequential decision-making mechanisms that adapt to adversarial probing and distributional drift.
  arXiv  Detail & Related papers  (2026-05-04T02:08:24Z)
• From Risk to Rescue: An Agentic Survival Analysis Framework for Liquidation Prevention [46.90931293070464]
  We propose an autonomous agent that perceives risk, simulates counterfactual futures, and executes protocol-faithful interventions to proactively prevent liquidations.<n>We validate our approach using a high-fidelity, protocol-faithful Aave v3 simulator on a cohort of 4,882 high-risk user profiles.
  arXiv  Detail & Related papers  (2026-04-16T03:22:05Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget [60.65072883773352]
  Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases.<n>We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute.<n>Our framework introduces an upper threshold that stops reasoning when the model is confident and a novel lower threshold that preemptively stops unsolvable instances.
  arXiv  Detail & Related papers  (2026-02-03T18:17:22Z)
• The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agents [37.75212140218036]
  We formalize the Loss-of-Control risk and identify the previously underexamined Intrinsic Value Misalignment (Intrinsic VM)<n>We then introduce IMPRESS, a scenario-driven framework for systematically assessing this risk.<n>We evaluate Intrinsic VM on 21 state-of-the-art LLM agents and find that it is a common and broadly observed safety risk across models.
  arXiv  Detail & Related papers  (2026-01-24T07:09:50Z)
• FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments [29.19862712902908]
  FinVault is the first execution-grounded security benchmark for financial agents.<n>We show that existing defense mechanisms remain ineffective in realistic financial agent settings.
  arXiv  Detail & Related papers  (2026-01-09T03:25:45Z)
• Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Audits [0.5586191108738564]
  We present a cross-market algorithmic trading system that balances execution quality with rigorous compliance enforcement.<n>The architecture comprises a high-level planner, a reinforcement learning execution agent, and an independent compliance agent.<n>We report effects at the 95% confidence level using paired t-tests and examine tail risk via CVaR.
  arXiv  Detail & Related papers  (2025-10-06T15:52:12Z)
• Uncovering the Vulnerability of Large Language Models in the Financial Domain via Risk Concealment [29.36824550283463]
  Large Language Models (LLMs) are increasingly integrated into financial applications, yet existing red-teaming research primarily targets harmful content.<n>We introduce Risk-Concealment Attacks (RCA), a novel multi-turn framework that iteratively conceals regulatory risks to provoke seemingly compliant yet regulatory-violating responses.<n>Experiments on FIN-Bench demonstrate that RCA effectively bypasses nine mainstream LLMs, achieving an average attack success rate (ASR) of 93.18%, including 98.28% on GPT-4.1 and 97.56% on OpenAI o1.
  arXiv  Detail & Related papers  (2025-09-07T22:35:15Z)
• SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents [58.21223208538351]
  This work explores the security issues surrounding mobile multimodal agents.<n>It attempts to construct a risk discrimination mechanism by incorporating behavioral sequence information.<n>It also designs an automated assisted assessment scheme based on a large language model.
  arXiv  Detail & Related papers  (2025-07-01T15:10:00Z)
• A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network [53.37983409425452]
  Large Language Models (LLMs) have achieved remarkable success across a wide range of applications.<n>Recently, collaborative frameworks such as the Multi-LLM Network (MultiLLMN) have been introduced.<n>We propose a novel Trusted MultiLLMN framework driven by a weighted Byzantine Fault Tolerance (WBFT) blockchain consensus mechanism.
  arXiv  Detail & Related papers  (2025-05-08T10:04:41Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.