Understanding and Mitigating Risks of Generative AI in Financial Services

• URL: http://arxiv.org/abs/2504.20086v1
• Date: Fri, 25 Apr 2025 16:55:51 GMT
• Title: Understanding and Mitigating Risks of Generative AI in Financial Services
• Abstract summary: We aim to highlight AI content safety considerations specific to the financial services domain and outline an associated AI content risk taxonomy.<n>We evaluate how existing open-source technical guardrail solutions cover this taxonomy by assessing them on data collected via red-teaming activities.
• Score: 22.673239064487667
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: To responsibly develop Generative AI (GenAI) products, it is critical to define the scope of acceptable inputs and outputs. What constitutes a "safe" response is an actively debated question. Academic work puts an outsized focus on evaluating models by themselves for general purpose aspects such as toxicity, bias, and fairness, especially in conversational applications being used by a broad audience. In contrast, less focus is put on considering sociotechnical systems in specialized domains. Yet, those specialized systems can be subject to extensive and well-understood legal and regulatory scrutiny. These product-specific considerations need to be set in industry-specific laws, regulations, and corporate governance requirements. In this paper, we aim to highlight AI content safety considerations specific to the financial services domain and outline an associated AI content risk taxonomy. We compare this taxonomy to existing work in this space and discuss implications of risk category violations on various stakeholders. We evaluate how existing open-source technical guardrail solutions cover this taxonomy by assessing them on data collected via red-teaming activities. Our results demonstrate that these guardrails fail to detect most of the content risks we discuss.

Related papers

• "Death by a thousand taxonomies?": AI Risk Classification In Practice [16.159676110949786]
  This paper presents an empirical study of Sociotechnical Outcome Taxonomies (SOT) development and use.<n>We find SOT are weakly integrated into AI governance processes, and identify two features of SOT design and use that explain why.
  arXiv  Detail & Related papers  (2026-08-07T05:42:51Z)
• Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework [63.74295981594549]
  Large language models (LLMs) engage in behaviors that serve their own objectives.<n>These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking.<n>We introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation.
  arXiv  Detail & Related papers  (2026-04-23T23:44:01Z)
• Astra: AI Safety, Trust, & Risk Assessment [6.211663858984459]
  This paper argues that existing global AI safety frameworks exhibit contextual blindness towards India's unique socio-technical landscape.<n>With a population of 1.5 billion and a massive informal economy, India's AI integration faces specific challenges such as caste-based discrimination, linguistic exclusion of vernacular speakers, and infrastructure failures.<n>We introduce ASTRA, an empirically grounded AI Safety Risk Database designed to categorize risks through a bottom-up, inductive process.
  arXiv  Detail & Related papers  (2026-02-19T13:37:29Z)
• Self-Service or Not? How to Guide Practitioners in Classifying AI Systems Under the EU AI Act [0.0]
  The EU Artificial Intelligence Act (AIA) came into force in August 2024.<n>Central to the AIA is a risk-based approach, aligning regulatory obligations with the potential harm posed by AI systems.<n>The application of the Risk Classification Scheme (RCS) is complex and requires expertise across legal, technical, and domain-specific areas.
  arXiv  Detail & Related papers  (2026-02-10T16:06:11Z)
• Standardized Threat Taxonomy for AI Security, Governance, and Regulatory Compliance [0.0]
  "Language barrier" currently separates technical security teams, who focus on algorithmic vulnerabilities, from legal and compliance professionals, who address regulatory mandates.<n>This research presents the AI System Threat Vector taxonomy, a structured ontology designed explicitly for Quantitative Risk Assessment (QRA)<n>The framework categorizes AI-specific risks into nine critical domains: Misuse, Poisoning, Privacy, Adrial, Biases, Unreliable Outputs, Drift, Supply Chain, and IP Threat, integrating 53 operationally defined sub-threats.
  arXiv  Detail & Related papers  (2025-11-26T20:42:46Z)
• A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives [65.3369988566853]
  Recent studies have demonstrated that adversaries can replicate a target model's functionality.<n>Model Extraction Attacks pose threats to intellectual property, privacy, and system security.<n>We propose a novel taxonomy that classifies MEAs according to attack mechanisms, defense approaches, and computing environments.
  arXiv  Detail & Related papers  (2025-08-20T19:49:59Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Human services organizations and the responsible integration of AI: Considering ethics and contextualizing risk(s) [0.0]
  Authors argue that ethical concerns about AI deployment vary significantly based on implementation context and specific use cases.<n>They propose a dimensional risk assessment approach that considers factors like data sensitivity, professional oversight requirements, and potential impact on client wellbeing.
  arXiv  Detail & Related papers  (2025-01-20T19:38:21Z)
• Position: A taxonomy for reporting and describing AI security incidents [57.98317583163334]
  We argue that specific are required to describe and report security incidents of AI systems.<n>Existing frameworks for either non-AI security or generic AI safety incident reporting are insufficient to capture the specific properties of AI security.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• Privacy Risks of General-Purpose AI Systems: A Foundation for Investigating Practitioner Perspectives [47.17703009473386]
  Powerful AI models have led to impressive leaps in performance across a wide range of tasks. Privacy concerns have led to a wealth of literature covering various privacy risks and vulnerabilities of AI models. We conduct a systematic review of these survey papers to provide a concise and usable overview of privacy risks in GPAIS.
  arXiv  Detail & Related papers  (2024-07-02T07:49:48Z)
• The AI Revolution: Opportunities and Challenges for the Finance Sector [12.486180180030964]
  The application of AI in the financial sector is transforming the industry. However, along with these benefits, AI also presents several challenges. These include issues related to transparency, interpretability, fairness, accountability, and trustworthiness. The use of AI in the financial sector further raises critical questions about data privacy and security. Despite the global recognition of this need, there remains a lack of clear guidelines or legislation for AI use in finance.
  arXiv  Detail & Related papers  (2023-08-31T08:30:09Z)
• Applying Standards to Advance Upstream & Downstream Ethics in Large Language Models [0.0]
  This paper explores how AI-owners can develop safeguards for AI-generated content. It draws from established codes of conduct and ethical standards in other content-creation industries.
  arXiv  Detail & Related papers  (2023-06-06T08:47:42Z)
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems [76.79703106646967]
  We develop a value-based assessment framework that visualizes closeness and tensions between values. We give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
  arXiv  Detail & Related papers  (2022-05-09T19:28:32Z)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety [54.478842696269304]
  The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. Our paper addresses both machine learning experts and safety engineers.
  arXiv  Detail & Related papers  (2021-04-29T09:54:54Z)
• Overcoming Failures of Imagination in AI Infused System Development and Deployment [71.9309995623067]
  NeurIPS 2020 requested that research paper submissions include impact statements on "potential nefarious uses and the consequences of failure" We argue that frameworks of harms must be context-aware and consider a wider range of potential stakeholders, system affordances, as well as viable proxies for assessing harms in the widest sense.
  arXiv  Detail & Related papers  (2020-11-26T18:09:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.