Doc2Oracle: Investigating the Impact of Javadoc Comments on Test Oracle Generation

• URL: http://arxiv.org/abs/2412.09360v1
• Date: Thu, 12 Dec 2024 15:27:47 GMT
• Title: Doc2Oracle: Investigating the Impact of Javadoc Comments on Test Oracle Generation
• Authors: Soneya Binta Hossain, Raygan Taylor, Matthew Dwyer,
• Abstract summary: In Java, Javadoc comments provide structured, natural language documentation embedded directly in the source code. We dive deep into investigating the impact of Javadoc comments on test oracle generation (TOG)
• Score: 0.716879432974126
• License:
• Abstract: Code documentation is a critical aspect of software development, serving as a bridge between human understanding and machine-readable code. Beyond assisting developers in understanding and maintaining code, documentation also plays a critical role in automating various software engineering tasks, such as test oracle generation (TOG). In Java, Javadoc comments provide structured, natural language documentation embedded directly in the source code, typically detailing functionality, usage, parameters, return values, and exceptions. While prior research has utilized Javadoc comments in test oracle generation (TOG), there has not been a thorough investigation into their impact when combined with other contextual information, nor into identifying the most relevant components for generating correct and strong test oracles, or understanding their role in detecting real bugs. In this study, we dive deep into investigating the impact of Javadoc comments on TOG.

Related papers

• CLOVER: A Test Case Generation Benchmark with Coverage, Long-Context, and Verification [71.34070740261072]
  This paper presents a benchmark, CLOVER, to evaluate models' capabilities in generating and completing test cases. The benchmark is containerized for code execution across tasks, and we will release the code, data, and construction methodologies.
  arXiv  Detail & Related papers  (2025-02-12T21:42:56Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch. Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests. Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• ASSERTIFY: Utilizing Large Language Models to Generate Assertions for Production Code [0.7973214627863593]
  Production assertions are statements embedded in the code to help developers validate their assumptions about the code. Current assertion generation techniques, such as static analysis and deep learning, fall short when it comes to generating production assertions. This preprint addresses the gap by introducing Assertify, an automated end-to-end tool that leverages Large Language Models (LLMs) and prompt engineering to generate production assertions.
  arXiv  Detail & Related papers  (2024-11-25T20:52:28Z)
• Generating executable oracles to check conformance of client code to requirements of JDK Javadocs using LLMs [21.06722050714324]
  This paper focuses on automation of test oracles for clients of widely used Java libraries, e.g., java.lang and java.util packages. We use large language models as an enabling technology to embody our insight into a framework for test oracle automation.
  arXiv  Detail & Related papers  (2024-11-04T04:24:25Z)
• CodeRAG-Bench: Can Retrieval Augment Code Generation? [78.37076502395699]
  We conduct a systematic, large-scale analysis of code generation using retrieval-augmented generation. We first curate a comprehensive evaluation benchmark, CodeRAG-Bench, encompassing three categories of code generation tasks. We examine top-performing models on CodeRAG-Bench by providing contexts retrieved from one or multiple sources.
  arXiv  Detail & Related papers  (2024-06-20T16:59:52Z)
• On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing [82.96523584351314]
  We decouple the task of context retrieval from the other components of the repository-level code editing pipelines. We conclude that while the reasoning helps to improve the precision of the gathered context, it still lacks the ability to identify its sufficiency.
  arXiv  Detail & Related papers  (2024-06-06T19:44:17Z)
• Prompting Code Interpreter to Write Better Unit Tests on Quixbugs Functions [0.05657375260432172]
  Unit testing is a commonly-used approach in software engineering to test the correctness and robustness of written code. In this study, we explore the effect of different prompts on the quality of unit tests generated by Code Interpreter. We find that the quality of the generated unit tests is not sensitive to changes in minor details in the prompts provided.
  arXiv  Detail & Related papers  (2023-09-30T20:36:23Z)
• LongCoder: A Long-Range Pre-trained Language Model for Code Completion [56.813974784131624]
  LongCoder employs a sliding window mechanism for self-attention and introduces two types of globally accessible tokens. Bridge tokens are inserted throughout the input sequence to aggregate local information and facilitate global interaction. memory tokens are included to highlight important statements that may be invoked later and need to be memorized.
  arXiv  Detail & Related papers  (2023-06-26T17:59:24Z)
• DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies [13.804337643709717]
  DocChecker is a tool for detecting and correcting differences between code and its accompanying comments. It is adept at identifying inconsistencies between code and comments, and it can also generate synthetic comments. It achieves a new State-of-the-art result of 72.3% accuracy on the Inconsistency Code-Comment Detection task.
  arXiv  Detail & Related papers  (2023-06-10T05:29:09Z)
• Using Developer Discussions to Guide Fixing Bugs in Software [51.00904399653609]
  We propose using bug report discussions, which are available before the task is performed and are also naturally occurring, avoiding the need for additional information from developers. We demonstrate that various forms of natural language context derived from such discussions can aid bug-fixing, even leading to improved performance over using commit messages corresponding to the oracle bug-fixing commits.
  arXiv  Detail & Related papers  (2022-11-11T16:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.