How Software Engineering Students Use LLMs to Write Research Papers: An Experience Report

• URL: http://arxiv.org/abs/2606.05114v2
• Date: Fri, 05 Jun 2026 20:32:05 GMT
• Title: How Software Engineering Students Use LLMs to Write Research Papers: An Experience Report
• Abstract summary: Large language models are increasingly becoming part of software engineering education.<n>This paper reports an educational experience involving the integration of reflective LLM use into an empirical methods assignment.
• Score: 3.415373522515852
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly becoming part of software engineering education, including activities involving empirical software engineering and evidence synthesis. This paper reports an educational experience involving the integration of reflective LLM use into an empirical methods assignment in a third-year software architecture course. Students were asked to develop a short research paper using either a rapid review or a gray literature review methodology and to disclose how LLMs were used throughout the assignment. We analyzed 146 student disclosure statements using a cross-analysis process combining LLM-assisted categorization with manual verification and refinement by the researchers. The reflections describe how students incorporated LLMs during activities such as brainstorming, methodological clarification, organization of findings, and writing refinement, while also reporting concerns regarding inaccuracies and verification of generated content. This experience report discusses lessons learned and educational implications for integrating AI-assisted technologies into empirical software engineering education.

Related papers

• Automated Summarization of Software Documents: An LLM-based Multi-Agent Approach [4.230025065044209]
  Large Language Models (LLMs) and Multi-Agent Systems (MAS) are revolutionizing software engineering (SE) by advancing automation, decision-making, and knowledge processing.<n>We present Metagente, an LLM-based MAS designed to generate concise and accurate summaries of software documentation.
  arXiv  Detail & Related papers  (2026-06-23T15:18:17Z)
• LLM Use, Cheating, and Academic Integrity in Software Engineering Education [2.6055991635575446]
  Cheating in university education is commonly described as context dependent and influenced by assessment design, institutional norms, and student interpretation.<n>This study investigates how software engineering students describe experiences of using large language models (LLMs) in ways they perceived as inappropriate, disallowed, or misaligned with course expectations.
  arXiv  Detail & Related papers  (2026-03-17T18:48:33Z)
• Testing the Untestable? An Empirical Study on the Testing Process of LLM-Powered Software Systems [0.0]
  This study explores how large language models are tested in the context of real-world application development.<n>Case study was conducted using 99 individual reports written by students who built and deployed LLM-powered applications as part of a university course.<n>Results: Testing LLM-powered systems required adaptations to traditional verification methods, blending source-level reasoning with behavior-aware evaluations.
  arXiv  Detail & Related papers  (2025-07-31T22:39:24Z)
• Can Large Language Models Help Students Prove Software Correctness? An Experimental Study with Dafny [75.55915044740566]
  Students in computing education increasingly use large language models (LLMs) such as ChatGPT.<n>This paper investigates how students interact with an LLM when solving formal verification exercises in Dafny.
  arXiv  Detail & Related papers  (2025-06-27T16:34:13Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective [64.00022624183781]
  Large language models (LLMs) can assess relevance and support information retrieval (IR) tasks.<n>We investigate how different LLM modules contribute to relevance judgment through the lens of mechanistic interpretability.
  arXiv  Detail & Related papers  (2025-04-10T16:14:55Z)
• A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models [50.34089812436633]
  Large Language Models (LLMs) have transformed natural language processing, yet their internal mechanisms remain largely opaque.<n> mechanistic interpretability has attracted significant attention from the research community as a means to understand the inner workings of LLMs.<n>Sparse Autoencoders (SAEs) have emerged as a promising method due to their ability to disentangle the complex, superimposed features within LLMs into more interpretable components.
  arXiv  Detail & Related papers  (2025-03-07T17:38:00Z)
• Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights [86.06371692309972]
  This work presents an analytical framework for the design and analysis of large language models (LLMs)-based algorithms.<n>Our proposed framework serves as an attempt to mitigate such headaches.
  arXiv  Detail & Related papers  (2024-07-20T07:39:07Z)
• Analyzing LLM Usage in an Advanced Computing Class in India [4.580708389528142]
  This study examines the use of large language models (LLMs) by undergraduate and graduate students for programming assignments in advanced computing classes. We conducted a comprehensive analysis involving 411 students from a Distributed Systems class at an Indian university.
  arXiv  Detail & Related papers  (2024-04-06T12:06:56Z)
• LLM Inference Unveiled: Survey and Roofline Model Insights [62.92811060490876]
  Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model. This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems.
  arXiv  Detail & Related papers  (2024-02-26T07:33:05Z)
• Towards an Understanding of Large Language Models in Software Engineering Tasks [29.30433406449331]
  Large Language Models (LLMs) have drawn widespread attention and research due to their astounding performance in text generation and reasoning tasks.<n>The evaluation and optimization of LLMs in software engineering tasks, such as code generation, have become a research focus.<n>This paper comprehensively investigate and collate the research and products combining LLMs with software engineering.
  arXiv  Detail & Related papers  (2023-08-22T12:37:29Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of supervised fine-tuning (SFT)<n>We also review the potential pitfalls of SFT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.