Integrating Large Language Models in Software Engineering Education: A Pilot Study through GitHub Repositories Mining

• URL: http://arxiv.org/abs/2509.04877v1
• Date: Fri, 05 Sep 2025 07:46:27 GMT
• Title: Integrating Large Language Models in Software Engineering Education: A Pilot Study through GitHub Repositories Mining
• Abstract summary: Large Language Models (LLMs) are increasingly adopted in software engineering (SE) education.<n>This research aims to develop a validated framework for integrating LLMs into SE education.
• Score: 1.043230260556633
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context: Large Language Models (LLMs) such as ChatGPT are increasingly adopted in software engineering (SE) education, offering both opportunities and challenges. Their adoption requires systematic investigation to ensure responsible integration into curricula. Objective: This doctoral research aims to develop a validated framework for integrating LLMs into SE education through a multi-phase process, including taxonomies development, empirical investigation, and case studies. This paper presents the first empirical step. Method: We conducted a pilot repository mining study of 400 GitHub projects, analyzing README files and issues discussions to identify the presence of motivator and demotivator previously synthesized in our literature review [ 8] study. Results: Motivators such as engagement and motivation (227 hits), software engineering process understanding (133 hits), and programming assistance and debugging support (97 hits) were strongly represented. Demotivators, including plagiarism and IP concerns (385 hits), security, privacy and data integrity (87 hits), and over-reliance on AI in learning (39 hits), also appeared prominently. In contrast, demotivators such as challenges in evaluating learning outcomes and difficulty in curriculum redesign recorded no hits across the repositories. Conclusion: The study provides early empirical validation of motivators/demotivators taxonomies with respect to their themes, highlights research practice gaps, and lays the foundation for developing a comprehensive framework to guide the responsible adoption of LLMs in SE education.

Related papers

• ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering [51.19819833971252]
  This paper introduces ECLAIR, a causally grounded AI framework that integrates Large Language Models into every stage of the scientific process.<n>ECLAIR treats LLMs as active scientific agents operating under the principles of causal inference.
  arXiv  Detail & Related papers  (2026-08-03T14:47:03Z)
• Structured AI Demonstrations and Student LLM Use in Engineering Mechanics: Study Design and Preliminary Results [0.29316801942271303]
  The rapid integration of large language models (LLMs) into undergraduate education presents an urgent challenge for engineering instructors.<n>Despite widespread student adoption, there remains a critical lack of domain-specific empirical evidence to guide pedagogical policies.<n>This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026.
  arXiv  Detail & Related papers  (2026-07-30T16:38:51Z)
• AI for Auto-Research: Roadmap & User Guide [107.0834449839233]
  We present an end-to-end analysis of AI across the complete research lifecycle.<n>We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy.<n>We show that greater automation can obscure rather than eliminate failure modes.
  arXiv  Detail & Related papers  (2026-05-18T17:08:26Z)
• A Discipline-Agnostic AI Literacy Course for Academic Research: Architecture, Pedagogy, and Implementation [0.0]
  The rapid integration of generative AI into academic literacy demands curricula that equip students not only with tool proficiency but with the critical judgment to use those tools responsibly in work offerings cluster around two poles: technical AI development courses serving specialist audiences, and brief general-literacy interventions that cannot develop sustained, practice-based competencies rigorous research requires.<n>This paper reports design, rationale, and implementation of a course at Lehigh University that addresses the underserved competencies required for rigorous AI-assisted literature review.
  arXiv  Detail & Related papers  (2026-04-29T21:47:00Z)
• EduResearchBench: A Hierarchical Atomic Task Decomposition Benchmark for Full-Lifecycle Educational Research [21.988207602041182]
  We introduce EduResearchBench, the first comprehensive evaluation platform dedicated to academic academic writing.<n>EduResearchBench is built upon our Hierarchical Atomic Task Decomposition (HATD) framework.<n>We propose a curriculum learning strategy that progressively builds competence from foundational skills to complex methodological reasoning and argumentation.
  arXiv  Detail & Related papers  (2026-01-22T09:52:30Z)
• Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models: Performance Benchmarking and Reasoning-Based Prompting Strategies [6.934935343001595]
  We extend prior curricular analytics research by examining a broader range of curriculum documents, competency frameworks, and models.<n>Using 7,600 manually annotated curriculum-competency alignment scores, we assess the informativeness of different curriculum sources.<n>We introduce a reasoning-based prompting strategy, Curricular CoT, to strengthen LLMs' pedagogical reasoning.
  arXiv  Detail & Related papers  (2026-01-16T04:07:23Z)
• Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey [59.3507264893654]
  Issue resolution is a complex Software Engineering task integral to real-world development.<n> benchmarks like SWE-bench revealed this task as profoundly difficult for large language models.<n>This paper presents a systematic survey of this emerging domain.
  arXiv  Detail & Related papers  (2026-01-15T18:55:03Z)
• From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews [37.98620195038937]
  We report a user study with researchers across multiple disciplines to characterize current practices, benefits, and textitpain points in using LLMs to investigate related work.<n>We identified three recurring gaps: (i) lack of trust in outputs, (ii) persistent verification burden, and (iii) requiring multiple tools.<n>This motivates our proposal of six design goals and a high-level framework that operationalizes them through improved related papers visualization, verification at every step, and human-feedback alignment with generation-guided explanations.
  arXiv  Detail & Related papers  (2025-12-12T15:38:34Z)
• Deep Research: A Systematic Survey [118.82795024422722]
  Deep Research (DR) aims to combine the reasoning capabilities of large language models with external tools, such as search engines.<n>This survey presents a comprehensive and systematic overview of deep research systems.
  arXiv  Detail & Related papers  (2025-11-24T15:28:28Z)
• Cognitive Foundations for Reasoning and Their Manifestation in LLMs [63.12951576410617]
  Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning.<n>We synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations.<n>We develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems.
  arXiv  Detail & Related papers  (2025-11-20T18:59:00Z)
• Integrating LLMs in Software Engineering Education: Motivators, Demotivators, and a Roadmap Towards a Framework for Finnish Higher Education Institutes [1.2825802100149328]
  The increasing adoption of Large Language Models (LLMs) in software engineering education presents both opportunities and challenges.<n>LLMs offer benefits such as enhanced learning experiences, automated assessments, and personalized tutoring.<n>Their integration also raises concerns about academic integrity, student over-reliance, and ethical considerations.
  arXiv  Detail & Related papers  (2025-03-28T08:41:43Z)
• The Lazy Student's Dream: ChatGPT Passing an Engineering Course on Its Own [3.2334676138718677]
  This paper presents a comprehensive investigation into the capability of Large Language Models (LLMs) to successfully complete a control systems course.<n>We assess LLM performance using ChatGPT under a "minimal effort" protocol that simulates realistic student usage patterns.<n>Our analysis provides quantitative insights into AI's strengths and limitations in handling mathematical formulations, coding challenges, and theoretical concepts in control systems engineering.
  arXiv  Detail & Related papers  (2025-02-23T18:47:14Z)
• Survey on Vision-Language-Action Models [0.2636873872510828]
  This work does not represent original research, but highlights how AI can help automate literature reviews.<n>Future research will focus on developing a structured framework for AI-assisted literature reviews.
  arXiv  Detail & Related papers  (2025-02-07T11:56:46Z)
• From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models [56.9134620424985]
  Cross-modal reasoning (CMR) is increasingly recognized as a crucial capability in the progression toward more sophisticated artificial intelligence systems. The recent trend of deploying Large Language Models (LLMs) to tackle CMR tasks has marked a new mainstream of approaches for enhancing their effectiveness. This survey offers a nuanced exposition of current methodologies applied in CMR using LLMs, classifying these into a detailed three-tiered taxonomy.
  arXiv  Detail & Related papers  (2024-09-19T02:51:54Z)
• SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories [55.161075901665946]
  Super aims to capture the realistic challenges faced by researchers working with Machine Learning (ML) and Natural Language Processing (NLP) research repositories. Our benchmark comprises three distinct problem sets: 45 end-to-end problems with annotated expert solutions, 152 sub problems derived from the expert set that focus on specific challenges, and 602 automatically generated problems for larger-scale development. We show that state-of-the-art approaches struggle to solve these problems with the best model (GPT-4o) solving only 16.3% of the end-to-end set, and 46.1% of the scenarios.
  arXiv  Detail & Related papers  (2024-09-11T17:37:48Z)
• Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
  Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
  arXiv  Detail & Related papers  (2024-07-17T20:01:21Z)
• Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents [80.5213198675411]
  Large language models (LLMs) have dramatically enhanced the field of language intelligence. LLMs leverage the intriguing chain-of-thought (CoT) reasoning techniques, obliging them to formulate intermediate steps en route to deriving an answer. Recent research endeavors have extended CoT reasoning methodologies to nurture the development of autonomous language agents.
  arXiv  Detail & Related papers  (2023-11-20T14:30:55Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.