The AI Assessment Scale Revisited: A Framework for Educational Assessment

• URL: http://arxiv.org/abs/2412.09029v1
• Date: Thu, 12 Dec 2024 07:44:52 GMT
• Title: The AI Assessment Scale Revisited: A Framework for Educational Assessment
• Authors: Mike Perkins, Jasper Roe, Leon Furze,
• Abstract summary: Recent developments in Generative Artificial Intelligence (GenAI) have created significant uncertainty in education. We present an updated version of the AI Assessment Scale (AIAS), a framework with two fundamental purposes.
• Score: 0.0
• License:
• Abstract: Recent developments in Generative Artificial Intelligence (GenAI) have created significant uncertainty in education, particularly in terms of assessment practices. Against this backdrop, we present an updated version of the AI Assessment Scale (AIAS), a framework with two fundamental purposes: to facilitate open dialogue between educators and students about appropriate GenAI use and to support educators in redesigning assessments in an era of expanding AI capabilities. Grounded in social constructivist principles and designed with assessment validity in mind, the AIAS provides a structured yet flexible approach that can be adapted across different educational contexts. Building on implementation feedback from global adoption across both the K-12 and higher education contexts, this revision represents a significant change from the original AIAS. Among these changes is a new visual guide that moves beyond the original traffic light system and utilises a neutral colour palette that avoids implied hierarchies between the levels. The scale maintains five distinct levels of GenAI integration in assessment, from "No AI" to "AI Exploration", but has been refined to better reflect rapidly advancing technological capabilities and emerging pedagogical needs. This paper presents the theoretical foundations of the revised framework, provides detailed implementation guidance through practical vignettes, and discusses its limitations and future directions. As GenAI capabilities continue to expand, particularly in multimodal content generation, the AIAS offers a starting point for reimagining assessment design in an era of disruptive technologies.

Related papers

• Dimensions of Generative AI Evaluation Design [51.541816010127256]
  We propose a set of general dimensions that capture critical choices involved in GenAI evaluation design. These dimensions include the evaluation setting, the task type, the input source, the interaction style, the duration, the metric type, and the scoring method.
  arXiv  Detail & Related papers  (2024-11-19T18:25:30Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) refers to the process of improving visual quality of images while removing degradation, such as noise, blur, weather effects, and so on. Traditional IR methods typically target specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions. The all-in-one image restoration (AiOIR) paradigm has emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• To accept or not to accept? An IRT-TOE Framework to Understand Educators' Resistance to Generative AI in Higher Education [0.0]
  This study aims to develop a theoretical model to empirically predict the barriers preventing educators from adopting Generative Artificial Intelligence in their classrooms. Our approach is grounded in the Innovation Resistance Theory (IRT) framework and augmented with constructs from the Technology-Organization-Environment (TOE) framework.
  arXiv  Detail & Related papers  (2024-07-29T15:59:19Z)
• Comprehensive AI Assessment Framework: Enhancing Educational Evaluation with Ethical AI Integration [0.0]
  This paper presents the Comprehensive AI Assessment Framework (CAIAF), an evolved version of the AI Assessment Scale (AIAS) by Perkins, Furze, Roe, and MacVaugh. The CAIAF incorporates stringent ethical guidelines, with clear distinctions based on educational levels, and advanced AI capabilities. The framework will ensure better learning outcomes, uphold academic integrity, and promote responsible use of AI.
  arXiv  Detail & Related papers  (2024-06-07T07:18:42Z)
• The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment [0.0]
  The rapid adoption of Generative Artificial Intelligence (GenAI) technologies in higher education has raised concerns about academic integrity, assessment practices, and student learning. This paper presents the findings of a pilot study conducted at British University Vietnam (BUV) exploring the implementation of the Artificial Intelligence Assessment Scale (AIAS) The AIAS consists of five levels, ranging from 'No AI' to 'Full AI', enabling educators to design assessments that focus on areas requiring human input and critical thinking.
  arXiv  Detail & Related papers  (2024-03-15T08:00:02Z)
• The AI Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment [0.0]
  We outline a practical, simple, and sufficiently comprehensive tool to allow for the integration of GenAI tools into educational assessment. The AI Assessment Scale (AIAS) empowers educators to select the appropriate level of GenAI usage in assessments. By adopting a practical, flexible approach, the AIAS can form a much-needed starting point to address the current uncertainty and anxiety regarding GenAI in education.
  arXiv  Detail & Related papers  (2023-12-12T09:08:36Z)
• How to Build an Adaptive AI Tutor for Any Course Using Knowledge Graph-Enhanced Retrieval-Augmented Generation (KG-RAG) [5.305156933641317]
  Large Language Models (LLMs) in Intelligent Tutoring Systems (ITS) presents transformative opportunities for personalized education. Current implementations face two critical challenges: maintaining factual accuracy and delivering coherent, context-aware instruction. This paper introduces Knowledge Graph-enhanced Retrieval-Augmented Generation (RAG), a novel framework that integrates structured knowledge representation with context-aware retrieval.
  arXiv  Detail & Related papers  (2023-11-29T15:02:46Z)
• Levels of AGI for Operationalizing Progress on the Path to AGI [64.59151650272477]
  We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI.
  arXiv  Detail & Related papers  (2023-11-04T17:44:58Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.