A Survey Instrument to Assess Students' AI and Generative AI Knowledge

• URL: http://arxiv.org/abs/2608.21391v1
• Date: Fri, 31 Jul 2026 17:18:10 GMT
• Title: A Survey Instrument to Assess Students' AI and Generative AI Knowledge
• Abstract summary: We present a survey that can be used to assess students' AI knowledge.<n> instrument combines measures from multiple scales to capture a range of literacy features and actual knowledge.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this research-to-practice paper we present a survey that can be used to assess students' AI knowledge. As the use of artificial intelligence (AI), including generative artificial intelligence (GenAI), has proliferated, so has the need to educate students about the topic. A range of AI literacy frameworks have been proposed, outlining the essential knowledge that students should have. Alongside, different ways of assessing AI knowledge have been developed. As yet, there is a lack of assessment instruments capable of evaluating multiple forms of student knowledge, including technical concepts, practical applications, and ethical concerns about AI use. In this article, we present a study implementing a comprehensive instrument to assess AI knowledge. The instrument combines measures from multiple scales to capture a range of literacy features and actual knowledge. We implemented the instrument in a higher education setting to assess its viability and usefulness and found that the instrument exhibited useful diagnostic capabilities and was able to identify common misconceptions among students. Although students performed well overall, there was a significant misunderstanding of how AI, especially GenAI systems, work. It also identified a lack of higher-level knowledge. The instrument is publicly available for use by others. We foresee its usefulness as a diagnostic that goes beyond understanding students' attitudes and perceptions of AI and GenAI use and tests multiple aspects of students' knowledge and conceptual understanding. This can enable the development of targeted instruction.

Related papers

• Exploring Student Perception on Gen AI Adoption in Higher Education: A Descriptive Study [1.978884131103313]
  This study examines how students perceive, use, and evaluate Generative Artificial Intelligence (GenAI)<n>Data were collected through a questionnaire administered to 436 postgraduate Computer Science students at the University of Hertfordshire.
  arXiv  Detail & Related papers  (2026-03-29T17:19:39Z)
• Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities [44.18936398140735]
  Article explores the use of AI tools to facilitate and enhance education.<n> Primers are provided on several core technical issues that arise when using AI in educational settings.<n>The article serves as a resource for researchers and educators seeking to advance AI's role in engineering education.
  arXiv  Detail & Related papers  (2025-09-10T19:19:26Z)
• Mapping Students' AI Literacy Framing and Learning through Reflective Journals [0.13154296174423616]
  This research paper presents a study of undergraduate technology students' self-reflective learning about artificial intelligence (AI)<n>It is important to understand what, how, and why students learn about AI so formal instruction can better support their learning.
  arXiv  Detail & Related papers  (2025-08-20T23:01:51Z)
• AI Literacy as a Key Driver of User Experience in AI-Powered Assessment: Insights from Socratic Mind [2.0272430076690027]
  This study examines how students' AI literacy and prior exposure to AI technologies shape their perceptions of Socratic Mind.<n>Data from 309 undergraduates in Computer Science and Business courses were collected.
  arXiv  Detail & Related papers  (2025-07-29T10:11:24Z)
• A Community-driven vision for a new Knowledge Resource for AI [54.6699845861346]
  Despite the success of knowledge resources like WordNet, verifiable, general-purpose widely available sources of knowledge remain a critical deficiency in AI infrastructure.<n>This paper synthesizes our findings and outlines a community-driven vision for a new knowledge infrastructure.
  arXiv  Detail & Related papers  (2025-06-19T20:51:28Z)
• Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications [0.2099922236065961]
  This study evaluates the Educational AI Hub, an AI-powered learning framework, implemented in undergraduate civil and environmental engineering courses at a large R1 public university.<n>Students valued the AI assistant for its accessibility and comfort, with nearly half reporting greater ease using it than seeking help from instructors or teaching assistants.<n>Ethical uncertainty, particularly around institutional policy and academic integrity, emerged as a key barrier to full engagement.
  arXiv  Detail & Related papers  (2025-06-06T03:02:49Z)
• AI Toolkit: Libraries and Essays for Exploring the Technology and Ethics of AI [0.0]
  The AITK project contains both Python libraries and computational essays (Jupyter notebooks)<n>These notebooks have been piloted at multiple institutions in a variety of humanities courses centered on the theme of responsible AI.<n>Pilot studies and usability testing results indicate that AITK is easy to navigate and effective at helping users gain a better understanding of AI.
  arXiv  Detail & Related papers  (2025-01-17T22:08:52Z)
• "From Unseen Needs to Classroom Solutions": Exploring AI Literacy Challenges & Opportunities with Project-based Learning Toolkit in K-12 Education [0.3994567502796064]
  There is a growing need to equip K-12 students with AI literacy skills that extend beyond computer science.<n>This paper explores the integration of a Project-Based Learning (PBL) AI toolkit into diverse subject areas, aimed at helping educators teach AI concepts more effectively.
  arXiv  Detail & Related papers  (2024-12-23T03:31:02Z)
• AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic Professionals [38.54324092761751]
  Generative AI has the potential to transform knowledge work, but further research is needed to understand how knowledge workers envision using and interacting with generative AI.<n>Our research focused on designing a generative AI assistant to aid genetic professionals in analyzing whole genome sequences (WGS) and other clinical data for rare disease diagnosis.
  arXiv  Detail & Related papers  (2024-12-19T22:54:49Z)
• Toward enriched Cognitive Learning with XAI [44.99833362998488]
  We introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by artificial intelligence (AI) tools. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle problems to enhance problem-solving skills.
  arXiv  Detail & Related papers  (2023-12-19T16:13:47Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)
• Thinking Fast and Slow in AI [38.8581204791644]
  This paper proposes a research direction to advance AI which draws inspiration from cognitive theories of human decision making. The premise is that if we gain insights about the causes of some human capabilities that are still lacking in AI, we may obtain similar capabilities in an AI system.
  arXiv  Detail & Related papers  (2020-10-12T20:10:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.