Knowledge-Based Version Incompatibility Detection for Deep Learning

• URL: http://arxiv.org/abs/2308.13276v2
• Date: Mon, 28 Aug 2023 14:13:54 GMT
• Title: Knowledge-Based Version Incompatibility Detection for Deep Learning
• Authors: Zhongkai Zhao, Bonan Kou, Mohamed Yilmaz Ibrahim, Muhao Chen, Tianyi Zhang
• Abstract summary: We propose to leverage the abundant discussions of DL version issues from Stack Overflow to facilitate version incompatibility detection. We reformulate the problem of knowledge extraction as a Question-Answering (QA) problem and use a pre-trained QA model to extract version compatibility knowledge.
• Score: 32.116361254082086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Version incompatibility issues are rampant when reusing or reproducing deep learning models and applications. Existing techniques are limited to library dependency specifications declared in PyPI. Therefore, these techniques cannot detect version issues due to undocumented version constraints or issues involving hardware drivers or OS. To address this challenge, we propose to leverage the abundant discussions of DL version issues from Stack Overflow to facilitate version incompatibility detection. We reformulate the problem of knowledge extraction as a Question-Answering (QA) problem and use a pre-trained QA model to extract version compatibility knowledge from online discussions. The extracted knowledge is further consolidated into a weighted knowledge graph to detect potential version incompatibilities when reusing a DL project. Our evaluation results show that (1) our approach can accurately extract version knowledge with 84% accuracy, and (2) our approach can accurately identify 65% of known version issues in 10 popular DL projects with a high precision (92%), while two state-of-the-art approaches can only detect 29% and 6% of these issues with 33% and 17% precision respectively.

Related papers

• PCREQ: Automated Inference of Compatible Requirements for Python Third-party Library Upgrades [5.857193811761703]
  Python third-party libraries (TPLs) are essential in modern software development, but upgrades often cause compatibility issues, leading to system failures.<n>Existing tools mainly detect dependency conflicts but overlook code-level incompatibilities.<n>We propose PCREQ, the first approach to automatically infer compatible requirements by combining version and code compatibility analysis.
  arXiv  Detail & Related papers  (2025-08-04T03:34:30Z)
• PropMEND: Hypernetworks for Knowledge Propagation in LLMs [82.99849359892112]
  We present a hypernetwork-based approach for knowledge propagation, named PropMEND.<n>We show almost 2x accuracy on challenging multi-hop questions whose answers are not explicitly stated in the injected fact.<n>We also introduce a new dataset, Controlled RippleEdit, to evaluate the generalization of our hypernetwork.
  arXiv  Detail & Related papers  (2025-06-10T15:44:19Z)
• Diverse Inference and Verification for Advanced Reasoning [19.88677753421871]
  Reasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding. We use a diverse inference approach that combines multiple models and methods at test time. We find that verifying mathematics and code problems, and rejection sampling on other problems is simple and effective.
  arXiv  Detail & Related papers  (2025-02-14T07:22:25Z)
• Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets [5.0091559832205155]
  We propose an automated source code autocuration technique to improve the quality of training data. We evaluate this method using The Stack v2 dataset, and find that 17% of the code versions in the dataset have newer versions. We expect our results to inspire process improvements for automated data curation, with the potential to enhance the reliability of outputs generated by AI tools.
  arXiv  Detail & Related papers  (2025-01-05T18:54:25Z)
• A Taxonomy of Self-Admitted Technical Debt in Deep Learning Systems [13.90991624629898]
  This paper empirically analyzes the presence of Self-Admitted Technical Debt (SATD) in Deep Learning systems. We derived a taxonomy of DL-specific SATD through open coding, featuring seven categories and 41 leaves.
  arXiv  Detail & Related papers  (2024-09-18T09:21:10Z)
• LLM-Enhanced Static Analysis for Precise Identification of Vulnerable OSS Versions [12.706661324384319]
  Open-source software (OSS) has experienced a surge in popularity, attributed to its collaborative development model and cost-effective nature. The adoption of specific software versions in development projects may introduce security risks when these versions bring along vulnerabilities. Current methods of identifying vulnerable versions typically analyze and trace the code involved in vulnerability patches using static analysis with pre-defined rules. This paper presents Vercation, an approach designed to identify vulnerable versions of OSS written in C/C++.
  arXiv  Detail & Related papers  (2024-08-14T06:43:06Z)
• Decide: Knowledge-Based Version Incompatibility Detection in Deep Learning Stacks [9.122391333975806]
  Version incompatibility issues are prevalent when reusing or reproducing deep learning (DL) models and applications. We present Decide, a web-based visualization of a knowledge graph that contains 2,376 version knowledge extracted from Stack Overflow discussions.
  arXiv  Detail & Related papers  (2024-08-04T20:05:22Z)
• A Large-scale Investigation of Semantically Incompatible APIs behind Compatibility Issues in Android Apps [13.24503570840706]
  We conduct a large-scale discovery of incompatible APIs in the Android Open Source Project (AOSP) We propose a unified framework to detect incompatible APIs, especially for semantic changes. Our approach detects 5,481 incompatible APIs spanning from version 4 to version 33.
  arXiv  Detail & Related papers  (2024-06-25T10:12:37Z)
• Enhancing Differential Testing With LLMs For Testing Deep Learning Libraries [8.779035160734523]
  This paper introduces an LLM-enhanced differential testing technique for DL libraries.<n>It addresses the challenges of finding alternative implementations for a given API and generating diverse test inputs.<n>It synthesizes counterparts for 1.84 times as many APIs as those found by state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-06-12T07:06:38Z)
• Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge [93.54427119091174]
  We propose outDated ISsue aware deCOding to enhance the performance of edited models on reasoning questions. We capture the difference in the probability distribution between the original and edited models. We amplify the difference of the token prediction in the edited model to alleviate the outdated issue.
  arXiv  Detail & Related papers  (2024-06-05T03:00:15Z)
• Bridging the Gap Between End-to-End and Two-Step Text Spotting [88.14552991115207]
  Bridging Text Spotting is a novel approach that resolves the error accumulation and suboptimal performance issues in two-step methods. We demonstrate the effectiveness of the proposed method through extensive experiments.
  arXiv  Detail & Related papers  (2024-04-06T13:14:04Z)
• Uncertainty-aware Language Modeling for Selective Question Answering [107.47864420630923]
  We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs. Our approach is model- and data-agnostic, is computationally-efficient, and does not rely on external models or systems.
  arXiv  Detail & Related papers  (2023-11-26T22:47:54Z)
• Automated Mapping of Vulnerability Advisories onto their Fix Commits in Open Source Repositories [7.629717457706326]
  We present an approach that combines practical experience and machine-learning (ML) An advisory record containing key information about a vulnerability is extracted from an advisory. A subset of candidate fix commits is obtained from the source code repository of the affected project.
  arXiv  Detail & Related papers  (2021-03-24T17:50:35Z)
• Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding [140.5911760063681]
  We propose a novel dataset named Knowledge-Routed Visual Question Reasoning for VQA model evaluation. We generate the question-answer pair based on both the Visual Genome scene graph and an external knowledge base with controlled programs.
  arXiv  Detail & Related papers  (2020-12-14T00:33:44Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.