Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment

• URL: http://arxiv.org/abs/2605.24837v1
• Date: Sun, 24 May 2026 03:06:34 GMT
• Title: Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment
• Abstract summary: Generative Artificial Intelligence (GenAI) presents a governance challenge for STEM assessment.<n>Unrestricted GenAI access enables task outsourcing that undermines the validity of traditional assessments.<n>This paper proposes a framework that treats GenAI as a design variable within the assessment argument rather than an external threat to it.
• Score: 4.027696535560039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Artificial Intelligence (GenAI) presents a governance challenge for STEM assessment. Unrestricted GenAI access enables task outsourcing that undermines the validity of traditional assessments; blanket prohibitions are difficult to enforce, may push use underground, and do little to prepare students for workplaces where GenAI-supported workflows are increasingly common. This paper addresses this dilemma by proposing a framework grounded in Evidence-Centered Design (ECD) that treats GenAI as a design variable within the assessment argument rather than an external threat to it. The framework analyzes how GenAI reshapes the student model, evidence model, and task model, and uses this analysis to articulate three principled governance stances. Restrict is warranted when GenAI would contaminate the inferential link between student work products and targeted unaided proficiency. Scaffold is warranted when bounded GenAI support can support peripheral demands without revealing the target construct, preserving inferential interpretability. Require is warranted when the target construct is disciplinary AI interaction competency and tasks can be designed to elicit process artifacts, including prompts, critiques, and revisions, that make student reasoning observable, scorable, and distinguishable from AI-generated output. This framework specifies when to restrict, scaffold, or require GenAI use in STEM assessment. We present two task designs deployed in an introductory physics course and demonstrate that disciplinary AI interaction competencies are observable in student response artifacts and can be scored using defensible rubrics grounded in student data and expert knowledge. By situating GenAI governance within validity arguments, the framework offers actionable guidance for preserving learning integrity while supporting authentic preparation for AI-enabled professional environments.

Related papers

• AI for Computational Design Science: A Responsible Human-AI Framework and Case Study on Short-Form Video Safety Surveillance [4.110053032708927]
  We develop AI for Computational Design Science (AI4CDS)<n>ChildRiskGuard is an interpretable artifact for detecting short-form videos inappropriate for children.<n>ChildRiskGuard achieves an F1 score of 0.769, substantially outperforming direct application of a general-purpose content-safety model.
  arXiv  Detail & Related papers  (2026-09-04T15:26:41Z)
• Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education [1.217503190366097]
  This paper contributes a design artefact and an accompanying position, developed for a Computer Science department on the basis of an existing taxonomy of GenAI usage.<n>By categorising AI usage across specific cognitive and developmental stages, the framework encourages students to reflect on their own learning process.
  arXiv  Detail & Related papers  (2026-06-11T14:18:34Z)
• How Do Software Engineering Students Use Generative AI in Real-World Capstone Projects? An Empirical Baseline Study [1.5102168012018966]
  The rapid integration of generative AI (GenAI) into professional software development adds new challenges.<n>We conducted a large-scale study of self-determined GenAI use in an undergraduate RWCP course.<n>Our findings provide a characterization of GenAI practices across the software engineering lifecycle.
  arXiv  Detail & Related papers  (2026-04-27T14:23:46Z)
• AI to Learn 2.0: A Deliverable-Oriented Governance Framework and Maturity Rubric for Opaque AI in Learning-Intensive Domains [0.0]
  Generative AI is entering research, education, and professional work faster than current governance frameworks can specify how AI-assisted outputs should be judged.<n>This paper proposes AI to Learn 2.0, a deliverable-oriented governance framework for AI-assisted work.
  arXiv  Detail & Related papers  (2026-03-16T11:44:43Z)
• A Design Science Blueprint for an Orchestrated AI Assistant in Doctoral Supervision [7.136113739927491]
  This study presents a design science blueprint for an orchestrated AI assistant and co-pilot in doctoral supervision that acts as a socio-technical mediator.<n>Risks in such a system exist, and among others, include AI over-reliance and the potential for the illusion of learning.<n>The contribution is an ex ante, literature-grounded design with workflow and governance rules that institutions can implement and trial across disciplines.
  arXiv  Detail & Related papers  (2025-10-22T04:20:35Z)
• Encouraging Students' Responsible Use of GenAI in Software Engineering Education: A Causal Model and Two Institutional Applications [0.4306143768014156]
  As generative AI (GenAI) tools become pervasive in education, concerns are rising about students using them to complete rather than learn.<n>This paper proposes and empirically applies a causal model to help educators scaffold responsible GenAI use in Software Engineering (SE) education.
  arXiv  Detail & Related papers  (2025-05-31T19:27:40Z)
• Enterprise Architecture as a Dynamic Capability for Scalable and Sustainable Generative AI adoption: Bridging Innovation and Governance in Large Organisations [55.2480439325792]
  Generative Artificial Intelligence is a powerful new technology with the potential to boost innovation and reshape governance in many industries.<n>However, organisations face major challenges in scaling GenAI, including technology complexity, governance gaps and resource misalignments.<n>This study explores how Enterprise Architecture Management can meet the complex requirements of GenAI adoption within large enterprises.
  arXiv  Detail & Related papers  (2025-05-09T07:41:33Z)
• Computational Safety for Generative AI: A Signal Processing Perspective [65.268245109828]
  computational safety is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI.<n>We show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts.<n>We discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.
  arXiv  Detail & Related papers  (2025-02-18T02:26:50Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption [0.0]
  We introduce "AI-Lab," a framework for guiding students in effectively leveraging GenAI within core programming courses. By identifying and rectifying GenAI's errors, students enrich their learning process. For educators, AI-Lab provides mechanisms to explore students' perceptions of GenAI's role in their learning experience.
  arXiv  Detail & Related papers  (2023-08-23T17:20:37Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.