ISTQB Certifications Under the Lens: Their Contributions to the Software-Testing Profession; and AI-assisted Synthesis of Practitioners' Endorsements and Criticisms

• URL: http://arxiv.org/abs/2603.14572v1
• Date: Sun, 15 Mar 2026 19:51:20 GMT
• Title: ISTQB Certifications Under the Lens: Their Contributions to the Software-Testing Profession; and AI-assisted Synthesis of Practitioners' Endorsements and Criticisms
• Abstract summary: We conducted an AI-assisted Multivocal Literature Review (MLR) to synthesize practitioner endorsements and criticisms.<n>We asked a panel of four independent experts to evaluate the precision of endorsements and fairness of criticisms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Objective: This study investigates the perceived value and critique of ISTQB certifications, the most widely recognized testing qualifications worldwide. While the certifications aim to standardize the software testing body of knowledge, debates persist about their practical relevance and impact. Our objective was to systematically capture practitioner perspectives and assess the precision of endorsements and fairness of criticisms through expert review. Method: We conducted an AI-assisted Multivocal Literature Review (MLR), combining academic and grey literature to synthesize practitioner endorsements (RQ1) and criticisms (RQ2). ChatGPT's deep research capability was employed under continuous human oversight, with QA strategies ensuring transparency and reliability. As another analysis, we asked a panel of four independent experts to evaluate the precision of endorsements and fairness of criticisms. Results: Practitioner endorsements emphasized career benefits, improved communication, and a shared vocabulary as the main values of ISTQB certifications. Criticisms focused on excessive theoretical content, limited relevance in agile and automation-intensive contexts, and weak support for real testing skills. Expert review confirmed that while many endorsements were precise, several criticisms reflected broader tensions in the discipline, including contrasting schools of thought in testing practice. Conclusions: ISTQB certifications provide recognizable career and communication value but remain contested in terms of practical utility. By triangulating practitioner voices with expert validation, this study delivers an evidence-based reflection on the strengths and weaknesses of ISTQB in shaping the software testing body of knowledge. The AI-assisted MLR also demonstrates how GenAI tools can support systematic evidence synthesis when coupled with rigorous human oversight.

Related papers

• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews [69.66583722746904]
  We introduce Beyond Rating, a holistic evaluation framework that assesses AI reviewers across five dimensions.<n>We propose a Max-Recall strategy to accommodate valid expert disagreement and introduce a curated dataset of paper with high-confidence reviews.<n>Our proposed text-centric metrics--particularly the recall of weakness arguments--correlate strongly with rating accuracy.
  arXiv  Detail & Related papers  (2026-04-21T14:21:15Z)
• Explainability and Certification of AI-Generated Educational Assessments [0.0]
  This chapter proposes a comprehensive framework for explainability and certification of AI-generated assessment items.<n>A structured certification metadata schema is introduced to capture provenance, alignment predictions, reviewer actions, and ethical indicators.<n>A proof-of-concept study on 500 AI-generated computer science questions demonstrates the framework's feasibility.
  arXiv  Detail & Related papers  (2026-03-18T11:33:58Z)
• Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education [0.27998963147546146]
  Existing AI literacy instruments, however, primarily target the general public, students, or K-12 teachers.<n>This study develops and validates the Faculty Artificial Intelligence Literacy and competency (FALCON-AI) Scale.
  arXiv  Detail & Related papers  (2026-03-03T23:20:33Z)
• Mirror: A Multi-Agent System for AI-Assisted Ethics Review [104.3684024153469]
  Mirror is an agentic framework for AI-assisted ethical review.<n>It integrates ethical reasoning, structured rule interpretation, and multi-agent deliberation within a unified architecture.
  arXiv  Detail & Related papers  (2026-02-09T03:38:55Z)
• Responsible Evaluation of AI for Mental Health [72.85175110624736]
  Current approaches to evaluating AI tools in mental health care are fragmented and poorly aligned with clinical practice, social context, and first-hand user experience.<n>This paper argues for a rethinking of responsible evaluation by introducing an interdisciplinary framework that integrates clinical soundness, social context, and equity.
  arXiv  Detail & Related papers  (2026-01-20T12:55:10Z)
• Escaping the Verifier: Learning to Reason via Demonstrations [0.4679031868857941]
  RARO learns strong reasoning capabilities from only expert demonstrations via Inverse Reinforcement Learning.<n>Our method sets up an adversarial interaction between a policy (generator) and a relativistic critic (discriminator)
  arXiv  Detail & Related papers  (2025-11-26T18:42:52Z)
• Expert Preference-based Evaluation of Automated Related Work Generation [54.29459509574242]
  We propose GREP, a multi-turn evaluation framework that integrates classical related work evaluation criteria with expert-specific preferences.<n>For better accessibility, we design two variants of GREP: a more precise variant with proprietary LLMs as evaluators, and a cheaper alternative with open-weight LLMs.
  arXiv  Detail & Related papers  (2025-08-11T13:08:07Z)
• Software Fairness Testing in Practice [0.21427777919040417]
  This study investigates how software professionals test AI-powered systems for fairness through interviews with 22 practitioners working on AI and ML projects.<n>Our findings highlight a significant gap between theoretical fairness concepts and industry practice.<n>Key challenges include data quality and diversity, time constraints, defining effective metrics, and ensuring model interoperability.
  arXiv  Detail & Related papers  (2025-06-20T16:03:02Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Measurement to Meaning: A Validity-Centered Framework for AI Evaluation [12.55408229639344]
  We provide a structured approach for reasoning about the types of evaluative claims that can be made given the available evidence.<n>Our framework is well-suited for the contemporary paradigm in machine learning.
  arXiv  Detail & Related papers  (2025-05-13T20:36:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.