A Novel Approach for Automated Design Information Mining from Issue Logs

• URL: http://arxiv.org/abs/2405.19623v1
• Date: Thu, 30 May 2024 02:20:04 GMT
• Title: A Novel Approach for Automated Design Information Mining from Issue Logs
• Authors: Jiuang Zhao, Zitian Yang, Li Zhang, Xiaoli Lian, Donghao Yang,
• Abstract summary: DRMiner is a novel method to automatically mine latent design rationales from developers' live discussion in open-source community. We acquire issue logs from Cassandra, Flink, and Solr repositories in Jira, and then annotate and process them under a rigorous scheme. DRMiner achieves an F1 score of 65% for mining design rationales, outperforming all baselines with a 7% improvement over GPT-4.0.
• Score: 3.5665328754813768
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Software architectures are usually meticulously designed to address multiple quality concerns and support long-term maintenance. However, due to the imbalance between the cost and value for developers to document design rationales (i.e., the design alternatives and the underlying arguments for making or rejecting decisions), these rationales are often obsolete or even missing. The lack of design knowledge has motivated a number of studies to extract design information from various platforms in recent years. Unfortunately, despite the wealth of discussion records related to design information provided by platforms like open-source communities, existing research often overlooks the underlying arguments behind alternatives due to challenges such as the intricate semantics of discussions and the lack of benchmarks for design rationale extraction. In this paper, we propose a novel method, named by DRMiner, to automatically mine latent design rationales from developers' live discussion in open-source community (i.e., issue logs in Jira). To better identify solutions and the arguments supporting them, DRMiner skillfully decomposes the problem into multiple text classification tasks and tackles them using prompt tuning of language models and customized text-related features. To evaluate DRMiner, we acquire issue logs from Cassandra, Flink, and Solr repositories in Jira, and then annotate and process them under a rigorous scheme, ultimately forming a dataset for design rationale mining. Experimental results show that DRMiner achieves an F1 score of 65% for mining design rationales, outperforming all baselines with a 7% improvement over GPT-4.0. Furthermore, we investigate the usefulness of the design rationales mined by DRMiner for automated program repair (APR) and find that the design rationales significantly enhance APR, achieving 14 times higher full-match repairs on average.

Related papers

• Reasoning Paths Optimization: Learning to Reason and Explore From Diverse Paths [69.39559168050923]
  We introduce Reasoning Paths Optimization (RPO), which enables learning to reason and explore from diverse paths. Our approach encourages favorable branches at each reasoning step while penalizing unfavorable ones, enhancing the model's overall problem-solving performance. We focus on multi-step reasoning tasks, such as math word problems and science-based exam questions.
  arXiv  Detail & Related papers  (2024-10-07T06:37:25Z)
• Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks [68.49251303172674]
  State-of-the-art large language models (LLMs) exhibit impressive problem-solving capabilities but may struggle with complex reasoning and factual correctness. Existing methods harness the strengths of chain-of-thought and retrieval-augmented generation (RAG) to decompose a complex problem into simpler steps and apply retrieval to improve factual correctness. We introduce Critic-guided planning with Retrieval-augmentation, CR-Planner, a novel framework that leverages fine-tuned critic models to guide both reasoning and retrieval processes through planning.
  arXiv  Detail & Related papers  (2024-10-02T11:26:02Z)
• Enhancing Automated Program Repair with Solution Design [5.547148114448699]
  We introduce DRCodePilot, an approach designed to augment GPT-4-Turbo's APR capabilities by incorporating DR into the prompt instruction. Our experimental results are impressive: DRCodePilot achieves a full-match ratio that is a remarkable 4.7x higher than when GPT-4 is utilized directly.
  arXiv  Detail & Related papers  (2024-08-22T01:13:02Z)
• Qualitative analysis of the relationship between design smells and software engineering challenges [3.9704849108478704]
  This research provides a tool which is used for design smell detection in Java software by analyzing large volume of source codes. Based on the output of the tool, a study is conducted to relate the cause of the detected design smells to two software engineering challenges namely "irregular team meetings" and "scope creep"
  arXiv  Detail & Related papers  (2023-10-22T23:21:13Z)
• Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning [27.224364543134094]
  We introduce a novel logic-driven data augmentation approach, AMR-LDA. AMR-LDA converts the original text into an Abstract Meaning Representation (AMR) graph. The modified AMR graphs are subsequently converted back into text to create augmented data.
  arXiv  Detail & Related papers  (2023-05-21T23:16:26Z)
• A Deep Learning Framework for Verilog Autocompletion Towards Design and Verification Automation [0.33598755777055367]
  This paper proposes a novel deep learning framework for training a Verilog autocompletion model. The framework involves integrating models pretrained on general programming language data and finetuning them on a dataset curated to be similar to a target downstream task. Experiments demonstrate that the proposed framework achieves better BLEU, ROUGE-L, and chrF scores by 9.5%, 6.7%, and 6.9%, respectively, compared to a model trained from scratch.
  arXiv  Detail & Related papers  (2023-04-26T21:56:03Z)
• Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration [71.95914457415624]
  Microprocessor architects are increasingly resorting to domain-specific customization in the quest for high-performance and energy-efficiency. We propose an alternative formulation that leverages Multi-Agent RL (MARL) to tackle this problem. Our evaluation shows that the MARL formulation consistently outperforms single-agent RL baselines.
  arXiv  Detail & Related papers  (2022-11-29T17:10:24Z)
• Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Matter [63.5550818034739]
  This paper presents a framework to evaluate state-of-the-art contributions to self-supervised monocular depth estimation. It includes pretraining, backbone, architectural design choices and loss functions. We re-implement, validate and re-evaluate 16 state-of-the-art contributions and introduce a new dataset.
  arXiv  Detail & Related papers  (2022-08-02T14:38:53Z)
• Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
  Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language. We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs. We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
  arXiv  Detail & Related papers  (2022-07-25T01:43:19Z)
• A survey on Variational Autoencoders from a GreenAI perspective [0.0]
  Variational AutoEncoders (VAEs) are powerful generative models that merge elements from statistics and information theory with the flexibility offered by deep neural networks. This article provides a comparative evaluation of some of the most successful, recent variations of VAEs.
  arXiv  Detail & Related papers  (2021-03-01T15:26:39Z)
• Semantic Graphs for Generating Deep Questions [98.5161888878238]
  We propose a novel framework which first constructs a semantic-level graph for the input document and then encodes the semantic graph by introducing an attention-based GGNN (Att-GGNN) On the HotpotQA deep-question centric dataset, our model greatly improves performance over questions requiring reasoning over multiple facts, leading to state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-04-27T10:52:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.