Metamorphic Debugging for Accountable Software

• URL: http://arxiv.org/abs/2409.16140v2
• Date: Tue, 22 Oct 2024 16:14:05 GMT
• Title: Metamorphic Debugging for Accountable Software
• Authors: Saeid Tizpaz-Niari, Shiva Darian, Ashutosh Trivedi,
• Abstract summary: Translating legalese into formal specifications is one challenge. Lack of a definitive 'truth' for queries (the oracle problem) is another. We propose that these challenges can be tackled by focusing on relational specifications.
• Score: 8.001739956625483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As the laws have become more complicated and enormous, the role of software systems in navigating and understanding these intricacies has become more critical. Given their socio-economic and legally critical implications, ensuring software accountability -- encompassing qualities such as legal compliance, explainability, perceptions of procedural justice, fairness of outcomes, and confidentiality/privacy -- is of paramount social importance. Moreover, software that accurately interprets its requirements, complies with legal standards and upholds social fairness can serve as a surrogate for legal and social norms, enabling policymakers to inquire about the law as seamlessly as a software engineer conducts a test. However, ensuring software accountability faces three key challenges: i) Translating legalese into formal specifications, ii) Lack of a definitive 'truth' for queries (the oracle problem), and iii) Scarcity of trustworthy datasets due to privacy and legal concerns. Drawing from the experiences in debugging U.S. tax preparation software, we propose that these challenges can be tackled by focusing on relational specifications. While the exact output for a given input may be unknown, the relationship between the outputs of two related inputs may be easier to express. This observation resembles i) the legal doctrine of precedent, meaning that similar cases must yield similar rulings; and ii) metamorphic relation (MR) in software engineering that requires a specific relation between software inputs and outputs. We propose metamorphic debugging as the foundation for detecting, explaining, and repairing socio-legal software for these relations. We showcase recent results that leverage metamorphic debugging to detect and explain accountability bugs in tax prep and poverty management software systems.

Related papers

• Ethical and Scalable Automation: A Governance and Compliance Framework for Business Applications [0.0]
  This paper introduces a framework ensuring that AI must be ethical, controllable, viable, and desirable. Different case studies validate this framework by integrating AI in both academic and practical environments.
  arXiv  Detail & Related papers  (2024-09-25T12:39:28Z)
• Towards Extracting Ethical Concerns-related Software Requirements from App Reviews [0.0]
  This study analyzes app reviews of the Uber mobile application (a popular taxi/ride app) We propose a novel approach that leverages a knowledge graph (KG) model to extract software requirements from app reviews. Our framework consists of three main components: developing an ontology with relevant entities and relations, extracting key entities from app reviews, and creating connections between them.
  arXiv  Detail & Related papers  (2024-07-19T04:50:32Z)
• Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making [69.44075077934914]
  "Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
  arXiv  Detail & Related papers  (2024-01-13T14:07:09Z)
• Legal Requirements Analysis [2.3349787245442966]
  We explore a variety of methods for analyzing legal requirements and exemplify them on representations. We describe possible alternatives for creating machine-analyzable representations from regulations.
  arXiv  Detail & Related papers  (2023-11-23T09:31:57Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• The right to audit and power asymmetries in algorithm auditing [68.8204255655161]
  We elaborate on the challenges and asymmetries mentioned by Sandvig at the IC2S2 2021. We also contribute a discussion of the asymmetries that were not covered by Sandvig. We discuss the implications these asymmetries have for algorithm auditing research.
  arXiv  Detail & Related papers  (2023-02-16T13:57:41Z)
• FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems [69.24490096929709]
  We developed an open source Python package called FAT Forensics. It can inspect important fairness, accountability and transparency aspects of predictive algorithms. Our toolbox can evaluate all elements of a predictive pipeline.
  arXiv  Detail & Related papers  (2022-09-08T13:25:02Z)
• Adversarial Scrutiny of Evidentiary Statistical Software [32.962815960406196]
  U.S. criminal legal system increasingly relies on software output to convict and incarcerate people. We propose robust adversarial testing as an audit framework to examine the validity of evidentiary statistical software.
  arXiv  Detail & Related papers  (2022-06-19T02:08:42Z)
• Metamorphic Testing and Debugging of Tax Preparation Software [2.185694185279913]
  We focus on an open-source tax preparation software for our case study. We develop a randomized test-case generation strategy to systematically validate the correctness of tax preparation software.
  arXiv  Detail & Related papers  (2022-05-10T16:10:10Z)
• Transparency, Compliance, And Contestability When Code Is(n't) Law [91.85674537754346]
  Both technical security mechanisms and legal processes serve as mechanisms to deal with misbehaviour according to a set of norms. While they share general similarities, there are also clear differences in how they are defined, act, and the effect they have on subjects. This paper considers the similarities and differences between both types of mechanisms as ways of dealing with misbehaviour.
  arXiv  Detail & Related papers  (2022-05-08T18:03:07Z)
• Out of Context: A New Clue for Context Modeling of Aspect-based Sentiment Analysis [54.735400754548635]
  ABSA aims to predict the sentiment expressed in a review with respect to a given aspect. The given aspect should be considered as a new clue out of context in the context modeling process. We design several aspect-aware context encoders based on different backbones.
  arXiv  Detail & Related papers  (2021-06-21T02:26:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.