Constant Optimization Driven Database System Testing

• URL: http://arxiv.org/abs/2501.11252v1
• Date: Mon, 20 Jan 2025 03:32:55 GMT
• Title: Constant Optimization Driven Database System Testing
• Authors: Chi Zhang, Manuel Rigger,
• Abstract summary: Logic bugs are bugs that can cause database management systems (DBMSs) to silently produce incorrect results for given queries. We propose Constant-Optimization-Driven Database Testing (CODDTest) as a novel approach for detecting logic bugs in databases.
• Score: 6.246028398098516
• License:
• Abstract: Logic bugs are bugs that can cause database management systems (DBMSs) to silently produce incorrect results for given queries. Such bugs are severe, because they can easily be overlooked by both developers and users, and can cause applications that rely on the DBMSs to malfunction. In this work, we propose Constant-Optimization-Driven Database Testing (CODDTest) as a novel approach for detecting logic bugs in DBMSs. This method draws inspiration from two well-known optimizations in compilers: constant folding and constant propagation. Our key insight is that for a certain database state and query containing a predicate, we can apply constant folding on the predicate by replacing an expression in the predicate with a constant, anticipating that the results of this predicate remain unchanged; any discrepancy indicates a bug in the DBMS. We evaluated CODDTest on five mature and extensively-tested DBMSs-SQLite, MySQL, CockroachDB, DuckDB, and TiDB-and found 45 unique, previously unknown bugs in them. Out of these, 24 are unique logic bugs. Our manual analysis of the state-of-the-art approaches indicates that 11 logic bugs are detectable only by CODDTest. We believe that CODDTest is easy to implement, and can be widely adopted in practice.

Related papers

• Finding Logic Bugs in Spatial Database Engines via Affine Equivalent Inputs [6.291508085458252]
  Spatial Database Management Systems (SDBMSs) aim to store, manipulate, and retrieve spatial data. The presence of logic bugs in SDBMSs can lead to incorrect results. Detecting logic bugs in SDBMSs is challenging due to the lack of ground truth for identifying incorrect results.
  arXiv  Detail & Related papers  (2024-10-16T12:18:16Z)
• DAC: Decomposed Automation Correction for Text-to-SQL [51.48239006107272]
  We introduce De Automation Correction (DAC), which corrects text-to-composed by decomposing entity linking and skeleton parsing. We show that our method improves performance by $3.7%$ on average of Spider, Bird, and KaggleDBQA compared with the baseline method.
  arXiv  Detail & Related papers  (2024-08-16T14:43:15Z)
• SQLaser: Detecting DBMS Logic Bugs with Clause-Guided Fuzzing [17.421408394486072]
  Database Management Systems (DBMSs) are vital components in modern data-driven systems. Their complexity often leads to logic bugs, which can lead to incorrect query results, data exposure, unauthorized access, etc. Existing detection employs two strategies: rule-based bug detection and coverage-guided fuzzing.
  arXiv  Detail & Related papers  (2024-07-05T06:56:33Z)
• Finding Cross-rule Optimization Bugs in Datalog Engines [8.849383195527627]
  We propose an automated testing approach called Incremental Rule Evaluation (IRE) IRE tackles the test oracle and test case generation problem. We implement IRE as a tool named Deopt and evaluate it on four Datalog engines.
  arXiv  Detail & Related papers  (2024-02-20T09:54:52Z)
• DebugBench: Evaluating Debugging Capability of Large Language Models [80.73121177868357]
  DebugBench is a benchmark for Large Language Models (LLMs) It covers four major bug categories and 18 minor types in C++, Java, and Python. We evaluate two commercial and four open-source models in a zero-shot scenario.
  arXiv  Detail & Related papers  (2024-01-09T15:46:38Z)
• Testing Database Engines via Query Plan Guidance [6.789710498230718]
  We propose the concept of Query Plan Guidance (QPG) for guiding automated testing towards "interesting" test cases. We apply our method to three mature, widely-used, and diverse database systems-DBite, TiDB, and Cockroach-and found 53 unique, previously unknown bugs.
  arXiv  Detail & Related papers  (2023-12-29T08:09:47Z)
• Detecting DBMS Bugs with Context-Sensitive Instantiation and Multi-Plan Execution [11.18715154222032]
  This paper aims to solve the two challenges, including how to generate semantically correctsql queries in a test case, and how to propose effective oracles to capture logic bugs. We have implemented a prototype system called Kangaroo and applied three widely used and well-tested semantic codes. The comparison between our system with the state-of-the-art systems shows that our system outperforms them in terms of the number of generated semantically valid queries, the explored code paths during testing, and the detected bugs.
  arXiv  Detail & Related papers  (2023-12-08T10:15:56Z)
• Teaching Large Language Models to Self-Debug [62.424077000154945]
  Large language models (LLMs) have achieved impressive performance on code generation. We propose Self- Debugging, which teaches a large language model to debug its predicted program via few-shot demonstrations.
  arXiv  Detail & Related papers  (2023-04-11T10:43:43Z)
• Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness [115.66421993459663]
  Recent studies reveal that text-to- models are vulnerable to task-specific perturbations. We propose a comprehensive robustness benchmark based on Spider to diagnose the model. We conduct a diagnostic study of the state-of-the-art models on the set.
  arXiv  Detail & Related papers  (2023-01-21T03:57:18Z)
• Using Developer Discussions to Guide Fixing Bugs in Software [51.00904399653609]
  We propose using bug report discussions, which are available before the task is performed and are also naturally occurring, avoiding the need for additional information from developers. We demonstrate that various forms of natural language context derived from such discussions can aid bug-fixing, even leading to improved performance over using commit messages corresponding to the oracle bug-fixing commits.
  arXiv  Detail & Related papers  (2022-11-11T16:37:33Z)
• Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database [86.03294330305097]
  We propose a unified semantic element for question answering (QA) on both knowledge bases (KB) and databases (DB) We introduce the primitive (relation and entity in KB, table name, column name and cell value in DB) as an essential element in our framework. We leverage the generator to predict final logical forms by altering and composing topranked primitives with different operations.
  arXiv  Detail & Related papers  (2022-11-09T19:33:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.