Higher education assessment practice in the era of generative AI tools

• URL: http://arxiv.org/abs/2404.01036v1
• Date: Mon, 1 Apr 2024 10:43:50 GMT
• Title: Higher education assessment practice in the era of generative AI tools
• Authors: Bayode Ogunleye, Kudirat Ibilola Zakariyyah, Oluwaseun Ajao, Olakunle Olayinka, Hemlata Sharma,
• Abstract summary: This study experimented using three assessment instruments from data science, data analytics, and construction management disciplines. Our findings revealed that GenAI tools exhibit subject knowledge, problem-solving, analytical, critical thinking, and presentation skills. Based on our findings, we made recommendations on how AI tools can be utilised for teaching and learning in HE.
• Score: 0.37282630026096586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The higher education (HE) sector benefits every nation's economy and society at large. However, their contributions are challenged by advanced technologies like generative artificial intelligence (GenAI) tools. In this paper, we provide a comprehensive assessment of GenAI tools towards assessment and pedagogic practice and, subsequently, discuss the potential impacts. This study experimented using three assessment instruments from data science, data analytics, and construction management disciplines. Our findings are two-fold: first, the findings revealed that GenAI tools exhibit subject knowledge, problem-solving, analytical, critical thinking, and presentation skills and thus can limit learning when used unethically. Secondly, the design of the assessment of certain disciplines revealed the limitations of the GenAI tools. Based on our findings, we made recommendations on how AI tools can be utilised for teaching and learning in HE.

Related papers

• Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge [78.35388859345056]
  We argue that the ML community would benefit from learning from and drawing on the social sciences when developing measurement instruments for evaluating GenAI systems. We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI.
  arXiv  Detail & Related papers  (2025-02-01T21:09:51Z)
• Exploring the Impact of Generative Artificial Intelligence in Education: A Thematic Analysis [0.7701938856931689]
  Large Language Models (LLMs) such as ChatGPT and Bard can be leveraged to automate boilerplate tasks. It is important to develop guidelines, policies, and assessment methods in the education sector to ensure the responsible integration of these tools.
  arXiv  Detail & Related papers  (2025-01-17T11:49:49Z)
• From Automation to Cognition: Redefining the Roles of Educators and Generative AI in Computing Education [2.0628700367476203]
  Generative Artificial Intelligence (GenAI) offers opportunities to revolutionise teaching and learning in Computing Education (CE) However, educators have expressed concerns that students may over-rely on GenAI and use these tools to generate solutions without engaging in the learning process. This paper describes our experiences with using GenAI in CS-focused educational settings and the changes we have implemented accordingly in our teaching.
  arXiv  Detail & Related papers  (2024-12-16T03:36:25Z)
• Dimensions of Generative AI Evaluation Design [51.541816010127256]
  We propose a set of general dimensions that capture critical choices involved in GenAI evaluation design. These dimensions include the evaluation setting, the task type, the input source, the interaction style, the duration, the metric type, and the scoring method.
  arXiv  Detail & Related papers  (2024-11-19T18:25:30Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• Understanding Student and Academic Staff Perceptions of AI Use in Assessment and Feedback [0.0]
  The rise of Artificial Intelligence (AI) and Generative Artificial Intelligence (GenAI) in higher education necessitates assessment reform. This study addresses a critical gap by exploring student and academic staff experiences with AI and GenAI tools. An online survey collected data from 35 academic staff and 282 students across two universities in Vietnam and one in Singapore.
  arXiv  Detail & Related papers  (2024-06-22T10:25:01Z)
• Crafting Tomorrow's Evaluations: Assessment Design Strategies in the Era of Generative AI [0.02638878351659022]
  GenAI has had an intense impact on education, significantly disrupting the assessment design and evaluation methodologies. There are several concerns primarily centred around academic integrity, authenticity, equity of access, assessment evaluation methodology, and feedback. This article discusses the challenges, and opportunities that need to be addressed for the assessment design and evaluation.
  arXiv  Detail & Related papers  (2024-05-03T01:28:21Z)
• The AI Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment [0.0]
  We outline a practical, simple, and sufficiently comprehensive tool to allow for the integration of GenAI tools into educational assessment. The AI Assessment Scale (AIAS) empowers educators to select the appropriate level of GenAI usage in assessments. By adopting a practical, flexible approach, the AIAS can form a much-needed starting point to address the current uncertainty and anxiety regarding GenAI in education.
  arXiv  Detail & Related papers  (2023-12-12T09:08:36Z)
• Tool Learning with Foundation Models [158.8640687353623]
  With the advent of foundation models, AI systems have the potential to be equally adept in tool use as humans. Despite its immense potential, there is still a lack of a comprehensive understanding of key challenges, opportunities, and future endeavors in this field.
  arXiv  Detail & Related papers  (2023-04-17T15:16:10Z)
• AI Explainability 360: Impact and Design [120.95633114160688]
  In 2019, we created AI Explainability 360 (Arya et al. 2020), an open source software toolkit featuring ten diverse and state-of-the-art explainability methods. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.
  arXiv  Detail & Related papers  (2021-09-24T19:17:09Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.