Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education

• URL: http://arxiv.org/abs/2411.19304v1
• Date: Thu, 28 Nov 2024 18:21:24 GMT
• Title: Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education
• Authors: Anamaria Mojica-Hanke, David Nader Palacio, Denys Poshyvanyk, Mario Linares-Vásquez, Steffen Herbold,
• Abstract summary: This study aims to contribute to the knowledge, about the synergy between Machine Learning (ML) and Software Engineering (SE) We analyzed SE researchers familiar with ML or who authored SE articles using ML, along with the articles themselves. We found diverse practices focusing on data collection, model training, and evaluation.
• Score: 12.716955305620191
• License:
• Abstract: Context: Machine Learning (ML) significantly impacts Software Engineering (SE), but studies mainly focus on practitioners, neglecting researchers. This overlooks practices and challenges in teaching, researching, or reviewing ML applications in SE. Objective: This study aims to contribute to the knowledge, about the synergy between ML and SE from the perspective of SE researchers, by providing insights into the practices followed when researching, teaching, and reviewing SE studies that apply ML. Method: We analyzed SE researchers familiar with ML or who authored SE articles using ML, along with the articles themselves. We examined practices, SE tasks addressed with ML, challenges faced, and reviewers' and educators' perspectives using grounded theory coding and qualitative analysis. Results: We found diverse practices focusing on data collection, model training, and evaluation. Some recommended practices (e.g., hyperparameter tuning) appeared in less than 20\% of literature. Common challenges involve data handling, model evaluation (incl. non-functional properties), and involving human expertise in evaluation. Hands-on activities are common in education, though traditional methods persist. Conclusion: Despite accepted practices in applying ML to SE, significant gaps remain. By enhancing guidelines, adopting diverse teaching methods, and emphasizing underrepresented practices, the SE community can bridge these gaps and advance the field.

Related papers

• MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs [97.94579295913606]
  Multimodal Large Language Models (MLLMs) have garnered increased attention from both industry and academia. In the development process, evaluation is critical since it provides intuitive feedback and guidance on improving models. This work aims to offer researchers an easy grasp of how to effectively evaluate MLLMs according to different needs and to inspire better evaluation methods.
  arXiv  Detail & Related papers  (2024-11-22T18:59:54Z)
• Surveying the MLLM Landscape: A Meta-Review of Current Surveys [17.372501468675303]
  Multimodal Large Language Models (MLLMs) have become a transformative force in the field of artificial intelligence. This survey aims to provide a systematic review of benchmark tests and evaluation methods for MLLMs.
  arXiv  Detail & Related papers  (2024-09-17T14:35:38Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Aligning Large Language Models with Human: A Survey [53.6014921995006]
  Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect information. This survey presents a comprehensive overview of these alignment technologies, including the following aspects.
  arXiv  Detail & Related papers  (2023-07-24T17:44:58Z)
• A Survey on Evaluation of Large Language Models [87.60417393701331]
  Large language models (LLMs) are gaining increasing popularity in both academia and industry. This paper focuses on three key dimensions: what to evaluate, where to evaluate, and how to evaluate.
  arXiv  Detail & Related papers  (2023-07-06T16:28:35Z)
• Towards machine learning guided by best practices [0.0]
  Machine learning (ML) is being used in software systems with multiple application fields, from medicine to software engineering (SE) This thesis aims to answer research questions that help to understand the practices used and discussed by practitioners and researchers in the SE community.
  arXiv  Detail & Related papers  (2023-04-29T10:58:37Z)
• What are the Machine Learning best practices reported by practitioners on Stack Exchange? [4.882319198853359]
  We present a study listing 127 Machine Learning best practices systematically mining 242 posts of 14 different Stack Exchange (STE) websites. The list of practices is presented in a set of categories related to different stages of the implementation process of an ML-enabled system.
  arXiv  Detail & Related papers  (2023-01-25T10:50:28Z)
• Machine Learning for Software Engineering: A Tertiary Study [13.832268599253412]
  Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009-2022, covering 6,117 primary studies. The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML.
  arXiv  Detail & Related papers  (2022-11-17T09:19:53Z)
• Lost in Translation: Reimagining the Machine Learning Life Cycle in Education [12.802237736747077]
  Machine learning (ML) techniques are increasingly prevalent in education. There is a pressing need to investigate how ML techniques support long-standing education principles and goals. In this work, we shed light on this complex landscape drawing on qualitative insights from interviews with education experts.
  arXiv  Detail & Related papers  (2022-09-08T17:14:01Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.