Where Assessment Validation and Responsible AI Meet

• URL: http://arxiv.org/abs/2411.02577v1
• Date: Mon, 04 Nov 2024 20:20:29 GMT
• Title: Where Assessment Validation and Responsible AI Meet
• Authors: Jill Burstein, Geoffrey T. LaFlair,
• Abstract summary: We propose a unified assessment framework that considers classical test validation theory and assessment-specific and domain-agnostic RAI principles and practice. The framework addresses responsible AI use for assessment that supports validity arguments, alignment with AI ethics to maintain human values and oversight, and broader social responsibility associated with AI use.
• Score: 0.0876953078294908
• License:
• Abstract: Validity, reliability, and fairness are core ethical principles embedded in classical argument-based assessment validation theory. These principles are also central to the Standards for Educational and Psychological Testing (2014) which recommended best practices for early applications of artificial intelligence (AI) in high-stakes assessments for automated scoring of written and spoken responses. Responsible AI (RAI) principles and practices set forth by the AI ethics community are critical to ensure the ethical use of AI across various industry domains. Advances in generative AI have led to new policies as well as guidance about the implementation of RAI principles for assessments using AI. Building on Chapelle's foundational validity argument work to address the application of assessment validation theory for technology-based assessment, we propose a unified assessment framework that considers classical test validation theory and assessment-specific and domain-agnostic RAI principles and practice. The framework addresses responsible AI use for assessment that supports validity arguments, alignment with AI ethics to maintain human values and oversight, and broader social responsibility associated with AI use.

Related papers

• Ethical AI Governance: Methods for Evaluating Trustworthy AI [0.552480439325792]
  Trustworthy Artificial Intelligence (TAI) integrates ethics that align with human values. TAI evaluation aims to ensure ethical standards and safety in AI development and usage.
  arXiv  Detail & Related papers  (2024-08-28T09:25:50Z)
• An evidence-based methodology for human rights impact assessment (HRIA) in the development of AI data-intensive systems [49.1574468325115]
  We show that human rights already underpin the decisions in the field of data use. This work presents a methodology and a model for a Human Rights Impact Assessment (HRIA) The proposed methodology is tested in concrete case-studies to prove its feasibility and effectiveness.
  arXiv  Detail & Related papers  (2024-07-30T16:27:52Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making [69.44075077934914]
  "Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
  arXiv  Detail & Related papers  (2024-01-13T14:07:09Z)
• Levels of AGI for Operationalizing Progress on the Path to AGI [64.59151650272477]
  We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI.
  arXiv  Detail & Related papers  (2023-11-04T17:44:58Z)
• Ethics in conversation: Building an ethics assurance case for autonomous AI-enabled voice agents in healthcare [1.8964739087256175]
  The principles-based ethics assurance argument pattern is one proposal in the AI ethics landscape. This paper presents the interim findings of a case study applying this ethics assurance framework to the use of Dora, an AI-based telemedicine system.
  arXiv  Detail & Related papers  (2023-05-23T16:04:59Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• A Framework for Ethical AI at the United Nations [0.0]
  This paper aims to provide an overview of the ethical concerns in artificial intelligence (AI) and the framework that is needed to mitigate those risks. It suggests a practical path to ensure the development and use of AI at the United Nations (UN) aligns with our ethical values.
  arXiv  Detail & Related papers  (2021-04-09T23:44:37Z)
• Enhanced well-being assessment as basis for the practical implementation of ethical and rights-based normative principles for AI [0.0]
  We propose the practical application of an enhanced well-being impact assessment framework for Autonomous and Intelligent Systems. This process could enable a human-centered algorithmically-supported approach to the understanding of the impacts of AI systems.
  arXiv  Detail & Related papers  (2020-07-29T13:26:05Z)
• Principles to Practices for Responsible AI: Closing the Gap [0.1749935196721634]
  We argue that an impact assessment framework is a promising approach to close the principles-to-practices gap. We review a case study of AI's use in forest ecosystem restoration, demonstrating how an impact assessment framework can translate into effective and responsible AI practices.
  arXiv  Detail & Related papers  (2020-06-08T16:04:44Z)
• On the Morality of Artificial Intelligence [154.69452301122175]
  We propose conceptual and practical principles and guidelines for Machine Learning research and deployment. We insist on concrete actions that can be taken by practitioners to pursue a more ethical and moral practice of ML aimed at using AI for social good.
  arXiv  Detail & Related papers  (2019-12-26T23:06:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.