Empirical Studies on Adversarial Reverse Engineering with Students

• URL: http://arxiv.org/abs/2603.03875v1
• Date: Wed, 04 Mar 2026 09:27:46 GMT
• Title: Empirical Studies on Adversarial Reverse Engineering with Students
• Abstract summary: This paper explores the use of students as participants in empirical reverse engineering experiments.<n>It examines their suitability and the necessary training; the design of appropriate challenges; strategies for ensuring the rigor and validity of the research.<n>We present a systematic literature review of existing reverse engineering experiments and user studies.
• Score: 24.226009059127374
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Empirical research in reverse engineering and software protection is crucial for evaluating the efficacy of methods designed to protect software against unauthorized access and tampering. However, conducting such studies with professional reverse engineers presents significant challenges, including access to professionals and affordability. This paper explores the use of students as participants in empirical reverse engineering experiments, examining their suitability and the necessary training; the design of appropriate challenges; strategies for ensuring the rigor and validity of the research and its results; ways to maintain students' privacy, motivation, and voluntary participation; and data collection methods. We present a systematic literature review of existing reverse engineering experiments and user studies, a discussion of related work from the broader domain of software engineering that applies to reverse engineering experiments, an extensive discussion of our own experience running experiments ourselves in the context of a master-level software hacking and protection course, and recommendations based on this experience. Our findings aim to guide future empirical studies in RE, balancing practical constraints with the need for meaningful, reproducible results.

Related papers

• Towards Improving the External Validity of Software Engineering Experiments with Transportability Methods [3.9236728810856714]
  We introduce transportability methods, their prerequisites, and demonstrate their potential through a simulation.<n>We then outline several SE research scenarios in which these methods could apply, e.g., how to effectively use students as substitutes for developers.<n>Finally, we outline a road map and practical guidelines to support SE researchers in applying them.
  arXiv  Detail & Related papers  (2026-04-09T12:57:26Z)
• The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research [56.80927148740585]
  We address the challenges of scalability and rigor by flipping the dynamic and developing AI agents as research evaluators.<n>We use mechanistic interpretability research as a testbed, build standardized research output, and develop MechEvalAgent.<n>Our work demonstrates the potential of AI agents to transform research evaluation and pave the way for rigorous scientific practices.
  arXiv  Detail & Related papers  (2026-02-05T19:00:02Z)
• Identity Theft in AI Conference Peer Review [50.18240135317708]
  We discuss newly uncovered cases of identity theft in the scientific peer-review process within artificial intelligence (AI) research.<n>We detail how dishonest researchers exploit the peer-review system by creating fraudulent reviewer profiles to manipulate paper evaluations.
  arXiv  Detail & Related papers  (2025-08-06T02:36:52Z)
• A Call for Critically Rethinking and Reforming Data Analysis in Empirical Software Engineering [5.687882380471718]
  Concerns about the correct application of empirical methodologies have existed since the 2006 Dagstuhl seminar on Empirical Software Engineering.<n>We conducted a literature survey of 27,000 empirical studies, using LLMs to classify statistical methodologies as adequate or inadequate.<n>We selected 30 primary studies and held a workshop with 33 ESE experts to assess their ability to identify and resolve statistical issues.
  arXiv  Detail & Related papers  (2025-01-22T09:05:01Z)
• Guiding Principles for Using Mixed Methods Research in Software Engineering [51.22583433491887]
  Mixed methods research is often used in software engineering, but researchers outside of the social or human sciences often lack experience when using these designs.<n>This paper provides guiding principles and advice on how to design mixed method research.
  arXiv  Detail & Related papers  (2024-04-09T04:34:25Z)
• Seeking Enlightenment: Incorporating Evidence-Based Practice Techniques in a Research Software Engineering Team [0.7340017786387767]
  Evidence-based practice (EBP) in software engineering aims to improve decision-making in software development by complementing practitioners' professional judgment with high-quality evidence from research. We believe the use of EBP techniques may be helpful for research software engineers (RSEs) in their work to bring software engineering best practices to scientific software development. We present an experience report on the use of a particular EBP technique, rapid reviews, within an RSE team at Sandia National Laboratories, and present practical recommendations for how to address barriers to EBP adoption within the RSE community.
  arXiv  Detail & Related papers  (2024-03-25T14:52:18Z)
• Are You a Real Software Engineer? Best Practices in Online Recruitment for Software Engineering Studies [4.247193377317027]
  Previous studies reported mixed outcomes and challenges in leveraging online platforms for the recruitment of qualified software engineers. We propose best practices for recruiting and screening participants to enhance the quality and relevance of both qualitative and quantitative software engineering (SE) research samples.
  arXiv  Detail & Related papers  (2024-02-02T21:53:28Z)
• Reusability Challenges of Scientific Workflows: A Case Study for Galaxy [56.78572674167333]
  This study examined the reusability of existing and exposed several challenges. The challenges preventing reusability include tool upgrading, tool support, design flaws, incomplete, failure to load a workflow, etc.
  arXiv  Detail & Related papers  (2023-09-13T20:17:43Z)
• Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning [65.12226891589592]
  This paper proposes a new approach to automated game validation and testing. Our method leverages a data-driven imitation learning technique, which requires little effort and time and no knowledge of machine learning or programming.
  arXiv  Detail & Related papers  (2022-08-15T11:08:44Z)
• Towards Understanding Barriers and Mitigation Strategies of Software Engineers with Non-traditional Educational and Occupational Backgrounds [3.1255188717445863]
  The traditional path to a software engineering career involves a post-secondary diploma in Software Engineering, Computer Science, or a related field. Many software engineers take a non-traditional path to their career, starting from other industries or fields of study. This paper proposes a study on barriers faced by software engineers with non-traditional educational and occupational backgrounds, and possible mitigation strategies for those barriers.
  arXiv  Detail & Related papers  (2022-04-08T22:51:19Z)
• A Field Guide to Federated Optimization [161.3779046812383]
  Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms.
  arXiv  Detail & Related papers  (2021-07-14T18:09:08Z)
• Open Source Software for Efficient and Transparent Reviews [0.11179881480027788]
  ASReview is an open source machine learning-aided pipeline applying active learning. We demonstrate by means of simulation studies that ASReview can yield far more efficient reviewing than manual reviewing.
  arXiv  Detail & Related papers  (2020-06-22T11:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.