Auto-assessment of assessment: A conceptual framework towards fulfilling the policy gaps in academic assessment practices

• URL: http://arxiv.org/abs/2411.08892v1
• Date: Mon, 28 Oct 2024 15:22:37 GMT
• Title: Auto-assessment of assessment: A conceptual framework towards fulfilling the policy gaps in academic assessment practices
• Authors: Wasiq Khan, Luke K. Topham, Peter Atherton, Raghad Al-Shabandar, Hoshang Kolivand, Iftikhar Khan, Abir Hussain,
• Abstract summary: We surveyed 117 academics from three countries (UK, UAE, and Iraq) We identified that most academics retain positive opinions regarding AI in education. For the first time, we propose a novel AI framework for autonomously evaluating students' work.
• Score: 4.770873744131964
• License:
• Abstract: Education is being transformed by rapid advances in Artificial Intelligence (AI), including emerging Generative Artificial Intelligence (GAI). Such technology can significantly support academics and students by automating monotonous tasks and making personalised suggestions. However, despite the potential of the technology, there are significant concerns regarding AI misuse, particularly by students in assessments. There are two schools of thought: one advocates for a complete ban on it, while the other views it as a valuable educational tool, provided it is governed by a robust usage policy. This contradiction clearly indicates a major policy gap in academic practices, and new policies are required to uphold academic standards while enabling staff and students to benefit from technological advancements. We surveyed 117 academics from three countries (UK, UAE, and Iraq), and identified that most academics retain positive opinions regarding AI in education. For example, the majority of experienced academics do not favour complete bans, and they see the potential benefits of AI for students, teaching staff, and academic institutions. Importantly, academics specifically identified the particular benefits of AI for autonomous assessment (71.79% of respondents agreed). Therefore, for the first time, we propose a novel AI framework for autonomously evaluating students' work (e.g., reports, coursework, etc.) and automatically assigning grades based on their knowledge and in-depth understanding of the submitted content. The survey results further highlight a significant lack of awareness of modern AI-based tools (e.g., ChatGPT) among experienced academics, a gap that must be addressed to uphold educational standards.

Related papers

• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• AI Governance in Higher Education: Case Studies of Guidance at Big Ten Universities [14.26619701452836]
  Generative AI has drawn significant attention from stakeholders in higher education. It simultaneously poses challenges to academic integrity and leads to ethical issues. Leading universities have already published guidelines on Generative AI. This study focuses on strategies for responsible AI governance as demonstrated in these guidelines.
  arXiv  Detail & Related papers  (2024-09-03T16:06:45Z)
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants [175.9723801486487]
  We evaluate whether two AI assistants, GPT-3.5 and GPT-4, can adequately answer assessment questions. GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions. Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
  arXiv  Detail & Related papers  (2024-08-07T12:11:49Z)
• Contra generative AI detection in higher education assessments [0.0]
  The rapid advancement and widespread adoption of generative AI, particularly in education, necessitates a reevaluation of traditional academic integrity mechanisms. We explore the effectiveness, vulnerabilities, and ethical implications of AI detection tools in the context of preserving academic integrity. This paper advocates for a strategic shift towards robust assessment methods and educational policies that embrace generative AI usage.
  arXiv  Detail & Related papers  (2023-12-08T18:46:44Z)
• The AI Incident Database as an Educational Tool to Raise Awareness of AI Harms: A Classroom Exploration of Efficacy, Limitations, & Future Improvements [14.393183391019292]
  The AI Incident Database (AIID) is one of the few attempts at offering a relatively comprehensive database indexing prior instances of harms or near harms stemming from the deployment of AI technologies in the real world. This study assesses the effectiveness of AIID as an educational tool to raise awareness regarding the prevalence and severity of AI harms in socially high-stakes domains.
  arXiv  Detail & Related papers  (2023-10-10T02:55:09Z)
• Is AI Changing the Rules of Academic Misconduct? An In-depth Look at Students' Perceptions of 'AI-giarism' [0.0]
  This study explores students' perceptions of AI-giarism, an emergent form of academic dishonesty involving AI and plagiarism. The findings portray a complex landscape of understanding, with clear disapproval for direct AI content generation. The study provides pivotal insights for academia, policy-making, and the broader integration of AI technology in education.
  arXiv  Detail & Related papers  (2023-06-06T02:22:08Z)
• New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution [2.94944680995069]
  ChatGPT's high performance on standardized academic tests has thrust the topic of artificial intelligence (AI) into the mainstream conversation about the future of education. This research aims to investigate the potential impact of AI on education through review and analysis of the existing literature across three major axes: applications, advantages, and challenges.
  arXiv  Detail & Related papers  (2023-05-12T08:22:54Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.