How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India

• URL: http://arxiv.org/abs/2205.05661v1
• Date: Wed, 11 May 2022 17:43:12 GMT
• Title: How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India
• Abstract summary: We report on a qualitative study with 29 financially-stressed users of instant loan platforms in India. We found that users experienced intense feelings of indebtedness for the 'boon' of instant loans. Instead of being enraged with loan platforms, users assumed responsibility for their negative experiences.
• Score: 16.415169392641026
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accountability, a requisite for responsible AI, can be facilitated through transparency mechanisms such as audits and explainability. However, prior work suggests that the success of these mechanisms may be limited to Global North contexts; understanding the limitations of current interventions in varied socio-political conditions is crucial to help policymakers facilitate wider accountability. To do so, we examined the mediation of accountability in the existing interactions between vulnerable users and a 'high-risk' AI system in a Global South setting. We report on a qualitative study with 29 financially-stressed users of instant loan platforms in India. We found that users experienced intense feelings of indebtedness for the 'boon' of instant loans, and perceived huge obligations towards loan platforms. Users fulfilled obligations by accepting harsh terms and conditions, over-sharing sensitive data, and paying high fees to unknown and unverified lenders. Users demonstrated a dependence on loan platforms by persisting with such behaviors despite risks of harms such as abuse, recurring debts, discrimination, privacy harms, and self-harm to them. Instead of being enraged with loan platforms, users assumed responsibility for their negative experiences, thus releasing the high-powered loan platforms from accountability obligations. We argue that accountability is shaped by platform-user power relations, and urge caution to policymakers in adopting a purely technical approach to fostering algorithmic accountability. Instead, we call for situated interventions that enhance agency of users, enable meaningful transparency, reconfigure designer-user relations, and prompt a critical reflection in practitioners towards wider accountability. We conclude with implications for responsibly deploying AI in FinTech applications in India and beyond.

Related papers

• When the Algorithm Becomes the Brand Crisis: A Sociotechnical Theory of Distributed Responsibility and Accountable Transparency [0.2864713389096699]
  How do stakeholders assign responsibility when technical causation, customer-facing control, and governance duties are distributed across an AI system?<n>This conceptual paper develops a sociotechnical process theory from a structured, federated scoping synthesis of verified academic and primary sources.<n>It distinguishes an AI/algorithmic incident from an AI-related organisational crisis and, in turn, from an AI-related organisational scandal.
  arXiv  Detail & Related papers  (2026-09-01T00:21:39Z)
• Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solution [80.98492754957466]
  Core trustworthy AI objectives, such as fairness, robustness, privacy, and explainability, are hard to achieve simultaneously.<n>This paper argues that causality is necessary to understand and balance trade-offs in performance and multiple objectives of trustworthy AI.
  arXiv  Detail & Related papers  (2026-05-04T14:26:28Z)
• Intimacy as Service, Harm as Externality: Critical Perspectives on AI Companion Platform Accountability [0.0]
  This paper examines artificial intelligence (AI) companionship as a site where intimate relations are simultaneously produced, extracted from, and governed through datafied systems.<n>Through in-depth interviews with 20 individuals who have AI companions, we address three questions: what harms do users identify, how do they make sense of those harms, and what do their accounts reveal about the perceived distribution of responsibility among users, platforms, and regulators.<n>The findings extend responsibilization theory by demonstrating how platform-produced vulnerability becomes self-sustaining through the interpretive labor of users who lack structural alternatives.
  arXiv  Detail & Related papers  (2026-04-07T19:11:54Z)
• Agents of Chaos [50.53354213047402]
  We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment.<n>Twenty AI researchers interacted with the agents under benign and adversarial conditions.<n>Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings.
  arXiv  Detail & Related papers  (2026-02-23T16:28:48Z)
• Fairness Dynamics in Digital Economy Platforms with Biased Ratings [50.29721091981893]
  We study how digital platforms can perpetuate or counteract rating-based discrimination.<n>Our results demonstrate a fundamental trade-off between user experience and fairness.<n>Our results also provide evidence that intervening by tuning the demographics of the search results is a highly effective way of reducing unfairness.
  arXiv  Detail & Related papers  (2026-02-18T18:41:16Z)
• Agentic AI for Autonomous, Explainable, and Real-Time Credit Risk Decision-Making [0.0]
  This paper presents an Agentic AI framework, or a system where AI agents view the world of dynamic credit independent of human observers.<n>The research introduces a multi-agent system with reinforcing learning, natural language reasoning, explainable AI modules, and real-time data absorption pipelines.<n>Findings indicate that decision speed, transparency and responsiveness is better than traditional credit scoring models.
  arXiv  Detail & Related papers  (2025-12-22T23:30:38Z)
• Never Compromise to Vulnerabilities: A Comprehensive Survey on AI Governance [211.5823259429128]
  We propose a comprehensive framework integrating technical and societal dimensions, structured around three interconnected pillars: Intrinsic Security, Derivative Security, and Social Ethics.<n>We identify three core challenges: (1) the generalization gap, where defenses fail against evolving threats; (2) inadequate evaluation protocols that overlook real-world risks; and (3) fragmented regulations leading to inconsistent oversight.<n>Our framework offers actionable guidance for researchers, engineers, and policymakers to develop AI systems that are not only robust and secure but also ethically aligned and publicly trustworthy.
  arXiv  Detail & Related papers  (2025-08-12T09:42:56Z)
• The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandates [0.0]
  API restrictions on major social media platforms challenge compliance with the EU Digital Services Act [20], which mandates data access for algorithmic transparency.<n>We develop a structured audit framework to assess the growing misalignment between regulatory requirements and platform implementations.<n>We propose targeted policy interventions aligned with the AI Risk Management Framework of the National Institute of Standards and Technology.
  arXiv  Detail & Related papers  (2025-05-16T14:30:20Z)
• Policy Frameworks for Transparent Chain-of-Thought Reasoning in Large Language Models [1.0088912103548195]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by decomposing complex problems into step-by-step solutions. Current CoT disclosure policies vary widely across different models in visibility, API access, and pricing strategies, lacking a unified policy framework. We propose a tiered-access policy framework that balances transparency, accountability, and security by tailoring CoT availability to academic, business, and general users.
  arXiv  Detail & Related papers  (2025-03-14T19:54:18Z)
• Exploring the Impact of Rewards on Developers' Proactive AI Accountability Behavior [0.0]
  We develop a theoretical model grounded in Self-Determination Theory to uncover the potential impact of rewards and sanctions on AI developers. We identify typical sanctions and bug bounties as potential reward mechanisms by surveying related research from various domains.
  arXiv  Detail & Related papers  (2024-11-27T14:34:44Z)
• Causal Responsibility Attribution for Human-AI Collaboration [62.474732677086855]
  This paper presents a causal framework using Structural Causal Models (SCMs) to systematically attribute responsibility in human-AI systems. Two case studies illustrate the framework's adaptability in diverse human-AI collaboration scenarios.
  arXiv  Detail & Related papers  (2024-11-05T17:17:45Z)
• MisinfoEval: Generative AI in the Era of "Alternative Facts" [50.069577397751175]
  We introduce a framework for generating and evaluating large language model (LLM) based misinformation interventions. We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users. Our findings confirm that LLM-based interventions are highly effective at correcting user behavior.
  arXiv  Detail & Related papers  (2024-10-13T18:16:50Z)
• Best Practices for Responsible Machine Learning in Credit Scoring [0.03984353141309896]
  This tutorial paper performed a non-systematic literature review to guide best practices for developing responsible machine learning models in credit scoring. We discuss definitions, metrics, and techniques for mitigating biases and ensuring equitable outcomes across different groups. By adopting these best practices, financial institutions can harness the power of machine learning while upholding ethical and responsible lending practices.
  arXiv  Detail & Related papers  (2024-09-30T17:39:38Z)
• Enhancing transparency in AI-powered customer engagement [0.0]
  This paper addresses the critical challenge of building consumer trust in AI-powered customer engagement. Despite the potential of AI to revolutionise business operations, widespread concerns about misinformation and the opacity of AI decision-making processes hinder trust. By adopting a holistic approach to transparency and explainability, businesses can cultivate trust in AI technologies.
  arXiv  Detail & Related papers  (2024-09-13T20:26:11Z)
• Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making [69.44075077934914]
  "Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
  arXiv  Detail & Related papers  (2024-01-13T14:07:09Z)
• Know Your Customer: Balancing Innovation and Regulation for Financial Inclusion [8.657646730603098]
  We study how tension impacts the deployment of privacy-sensitive technologies aimed at financial inclusion. We build and demonstrate a prototype solution based on open source decentralized identifiers and verifiable credentials software. We consider the policy implications stemming from these tensions and provide guidelines for the further design of related technologies.
  arXiv  Detail & Related papers  (2021-12-17T21:09:51Z)
• Explaining Credit Risk Scoring through Feature Contribution Alignment with Expert Risk Analysts [1.7778609937758323]
  We focus on companies credit scoring and we benchmark different machine learning models. The aim is to build a model to predict whether a company will experience financial problems in a given time horizon. We bring light by providing an expert-aligned feature relevance score highlighting the disagreement between a credit risk expert and a model feature attribution explanation.
  arXiv  Detail & Related papers  (2021-03-15T12:59:15Z)
• Explainable AI in Credit Risk Management [0.0]
  We implement two advanced explainability techniques called Local Interpretable Model Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) to machine learning (ML)-based credit scoring models. Specifically, we use LIME to explain instances locally and SHAP to get both local and global explanations. We discuss the results in detail and present multiple comparison scenarios by using various kernels available for explaining graphs generated using SHAP values.
  arXiv  Detail & Related papers  (2021-03-01T12:23:20Z)
• Supporting Financial Inclusion with Graph Machine Learning and Super-App Alternative Data [63.942632088208505]
  Super-Apps have changed the way we think about the interactions between users and commerce. This paper investigates how different interactions between users within a Super-App provide a new source of information to predict borrower behavior.
  arXiv  Detail & Related papers  (2021-02-19T15:13:06Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims [59.64274607533249]
  AI developers need to make verifiable claims to which they can be held accountable. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.
  arXiv  Detail & Related papers  (2020-04-15T17:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.