Four Years of GenAI: How Educators and Industry Adapted Their Assessment Strategies

• URL: http://arxiv.org/abs/2607.24246v2
• Date: Tue, 28 Jul 2026 11:56:51 GMT
• Title: Four Years of GenAI: How Educators and Industry Adapted Their Assessment Strategies
• Abstract summary: We investigate whether the core skills for entry-level engineers have changed with GenAI.<n>We draw on responses from 56 educators and 24 hiring professionals across diverse geographic regions.<n>We find that educators frame their GenAI policies primarily around academic integrity, whereas GenAI use in industry is near-universal.
• Score: 2.5306843392410023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: GenAI's ability to solve a wide range of software engineering tasks is reshaping the software industry, raising the question of what an entry-level software engineer looks like today. In this work, we investigate whether the core skills for entry-level engineers have changed with GenAI and how the assessment of those skills has changed. Through two surveys, we consider both the educational and hiring perspectives, drawing on responses from 56 educators and 24 hiring professionals across diverse geographic regions to identify gaps and misaligned expectations. We find that educators frame their GenAI policies primarily around academic integrity, whereas GenAI use in industry is near-universal, with organizational policy governing how it should be used rather than whether to use it. Policies on GenAI use during interviews are still emerging, and respondents are split on whether they prefer candidates who demonstrate GenAI skills. Both populations are moving toward GenAI-resistant assessments built on observable real-time interactions and higher-order tasks, and both agree on which skills matter most, rating critical evaluation of AI-generated output, responsible and effective use of GenAI tools, and the ability to learn and adapt independently highly. However, they disagree on graduate readiness in these skills: hiring professionals report larger gaps, with the disagreement concentrated on foundational, non-GenAI-specific skills.

Related papers

• Quick Build, Careful Check? Generative AI Use in Hackathons [83.0493885475068]
  We report an exploratory interview study with participants at an AI-themed hackathon in central Europe.<n>We found that participants used GenAI for purposes beyond coding, including learning unfamiliar topics, brainstorming, and preparing documentation.<n>We also found that all participants converged on an unwritten practice of checking GenAI output before using it, yet their ability to actually verify was constrained by time pressure and limited domain knowledge.
  arXiv  Detail & Related papers  (2026-07-31T08:59:18Z)
• How Do Software Engineering Students Use Generative AI in Real-World Capstone Projects? An Empirical Baseline Study [1.5102168012018966]
  The rapid integration of generative AI (GenAI) into professional software development adds new challenges.<n>We conducted a large-scale study of self-determined GenAI use in an undergraduate RWCP course.<n>Our findings provide a characterization of GenAI practices across the software engineering lifecycle.
  arXiv  Detail & Related papers  (2026-04-27T14:23:46Z)
• Product Manager Practices for Delegating Work to Generative AI: "Accountability must not be delegated to non-human actors" [14.99180726933919]
  Generative AI (GenAI) is changing the nature of knowledge work, particularly for Product Managers (PMs) in software development teams.<n>We conducted a mixed-methods study at Microsoft, a large, multinational software company.<n>We contribute: (1) PMs' current GenAI adoption rates, uses cases, and perceived benefits and barriers; (2) a framework capturing how PMs assess which tasks to delegate to GenAI; and (3) PMs adaptation practices for integrating GenAI into their roles and perceptions of how their role is evolving.
  arXiv  Detail & Related papers  (2025-10-02T19:19:49Z)
• Encouraging Students' Responsible Use of GenAI in Software Engineering Education: A Causal Model and Two Institutional Applications [0.4306143768014156]
  As generative AI (GenAI) tools become pervasive in education, concerns are rising about students using them to complete rather than learn.<n>This paper proposes and empirically applies a causal model to help educators scaffold responsible GenAI use in Software Engineering (SE) education.
  arXiv  Detail & Related papers  (2025-05-31T19:27:40Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• Enterprise Architecture as a Dynamic Capability for Scalable and Sustainable Generative AI adoption: Bridging Innovation and Governance in Large Organisations [55.2480439325792]
  Generative Artificial Intelligence is a powerful new technology with the potential to boost innovation and reshape governance in many industries.<n>However, organisations face major challenges in scaling GenAI, including technology complexity, governance gaps and resource misalignments.<n>This study explores how Enterprise Architecture Management can meet the complex requirements of GenAI adoption within large enterprises.
  arXiv  Detail & Related papers  (2025-05-09T07:41:33Z)
• The GenAI Generation: Student Views of Awareness, Preparedness, and Concern [1.5709900716890133]
  Outpacing the development of uniform policies and structures, GenAI has heralded a unique era and given rise to the GenAI Generation.<n>This study examines students' perceptions of GenAI through a concise survey with optional open-ended questions.<n>Students with greater curricular exposure to GenAI tend to feel more prepared, while those without it more often express vulnerability and uncertainty.
  arXiv  Detail & Related papers  (2025-05-04T19:37:13Z)
• Lessons for GenAI Literacy From a Field Study of Human-GenAI Augmentation in the Workplace [0.11704154007740832]
  This study compares the use of GenAI across three different functions - product development, software engineering, and digital content creation.<n>Findings show a wide variance in the use of GenAI and in the level of computing knowledge of users.
  arXiv  Detail & Related papers  (2025-02-01T21:26:31Z)
• Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge [78.35388859345056]
  We argue that the ML community would benefit from learning from and drawing on the social sciences when developing measurement instruments for evaluating GenAI systems.<n>We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI systems.
  arXiv  Detail & Related papers  (2025-02-01T21:09:51Z)
• Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023 [38.83649319653387]
  There is limited prior research on computing students' use and perceptions of GenAI. We surveyed all computer science majors in a small engineering-focused R1 university. We discuss the impact of our findings on the emerging conversation around GenAI and education.
  arXiv  Detail & Related papers  (2024-11-17T20:17:47Z)
• Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI [41.96102438774773]
  This work presents the findings from a university-level competition, which challenged participants to design prompts for eliciting biased outputs from GenAI tools. We quantitatively and qualitatively analyze the competition submissions and identify a diverse set of biases in GenAI and strategies employed by participants to induce bias in GenAI.
  arXiv  Detail & Related papers  (2024-10-20T18:44:45Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• Teacher agency in the age of generative AI: towards a framework of hybrid intelligence for learning design [0.0]
  Generative AI (genAI) is being used in education for different purposes. From the teachers' perspective, genAI can support activities such as learning design. However, GenAI has the potential to negatively affect professional agency due to teachers' limited power.
  arXiv  Detail & Related papers  (2024-07-09T08:28:05Z)
• Investigating Explainability of Generative AI for Code through Scenario-based Design [44.44517254181818]
  generative AI (GenAI) technologies are maturing and being applied to application domains such as software engineering. We conduct 9 workshops with 43 software engineers in which real examples from state-of-the-art generative AI models were used to elicit users' explainability needs. Our work explores explainability needs for GenAI for code and demonstrates how human-centered approaches can drive the technical development of XAI in novel domains.
  arXiv  Detail & Related papers  (2022-02-10T08:52:39Z)
• Creation and Evaluation of a Pre-tertiary Artificial Intelligence (AI) Curriculum [58.86139968005518]
  The Chinese University of Hong Kong (CUHK)-Jockey Club AI for the Future Project (AI4Future) co-created an AI curriculum for pre-tertiary education. A team of 14 professors with expertise in engineering and education collaborated with 17 principals and teachers from 6 secondary schools to co-create the curriculum. The co-creation process generated a variety of resources which enhanced the teachers knowledge in AI, as well as fostered teachers autonomy in bringing the subject matter into their classrooms.
  arXiv  Detail & Related papers  (2021-01-19T11:26:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.