DCI: Dependency Confidence Index for Assessing Open-Source Dependency Trustworthiness

• URL: http://arxiv.org/abs/2608.16430v1
• Date: Mon, 17 Aug 2026 11:31:03 GMT
• Title: DCI: Dependency Confidence Index for Assessing Open-Source Dependency Trustworthiness
• Abstract summary: We present the Dependency Confidence Index (DCI), a composite formative index that combines nine empirically weighted trust factors into a single normalized composite score for dependency selection.<n>We conducted a pilot evaluation of the normalized DCI on 92 popular PyPI packages, observing moderate agreement with OpenSSF Scorecard scores and perfect test-retest reliability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Selecting trustworthy open source software dependencies remains a major challenge in software supply chain security. We present the Dependency Confidence Index (DCI), a composite formative index that combines nine empirically weighted trust factors into a single normalized composite score for dependency selection. DCI's trust factors combine insights from a systematic literature review and an exploratory Analytic Hierarchy Process (AHP) survey of ten software developers, highlighting security, source code quality, and project health as the most influential dimensions. Following Goal-Question-Metric methodology, we implemented 12 automated measurements using SonarQube, GitHub APIs, and OpenSSF Scorecard data, deployed in a containerized evaluation platform. We conducted a pilot evaluation of the normalized DCI on 92 popular PyPI packages, observing moderate agreement with OpenSSF Scorecard scores and perfect test--retest reliability. Analysis reveals process-based factors (dependency management, CI) dominate scores on high-quality packages, while security metrics saturate---suggesting DCI's complementary role to existing tools. Our publicly available implementation provides a foundation for open source software trustworthiness research and practical dependency auditing.

Related papers

• Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management [0.7329200485567826]
  In software engineering, multi-agent AI systems are increasingly used to automate requirements analysis, architecture design, test generation, and traceability linking.<n>We propose a trust-aware coordination framework where a shared knowledge graph serves as both centralized semantic memory and a coordination surface.<n>We evaluate on an automotive software engineering case study measuring link prediction calibration, protocol effectiveness, threshold sensitivity, and the impact of traceability seeding.
  arXiv  Detail & Related papers  (2026-06-15T18:41:50Z)
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security [57.35851886874902]
  Agentic AI systems execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness.<n>This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments.<n>For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies.
  arXiv  Detail & Related papers  (2026-05-17T10:26:37Z)
• Leveraging Large Language Models for Trustworthiness Assessment of Web Applications [13.909850314037653]
  This study presents an empirical methodology to automate the trustworthiness assessment of web applications by leveraging Large Language Models (LLMs)<n>We propose an extension of a hierarchical Quality Model (QM) based on the Logic Score of Preference (LSP)<n> Experimental results indicate that excessive structural context can introduce noise, whereas rule-based instructional prompting improves assessment reliability.
  arXiv  Detail & Related papers  (2026-03-24T23:33:54Z)
• Decision Quality Evaluation Framework at Pinterest [0.36944296923226316]
  The framework is centered on a high-trust Golden Set (GDS) curated by subject matter experts (SMEs)<n>We introduce an automated intelligent sampling pipeline that uses propensity scores to efficiently expand dataset coverage.<n>The framework enables a shift from subjective assessments to a data-driven and quantitative practice for managing content safety systems.
  arXiv  Detail & Related papers  (2026-02-17T18:45:55Z)
• Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems [94.9141394384021]
  Individual agents in multi-agent systems often lack robustness, tending to blindly conform to misleading peers.<n>We show this weakness stems from both sycophancy and inadequate ability to evaluate peer reliability.<n>We first formalize the learning problem of history-aware reference, introducing the historical interactions of peers as additional input.<n>We then develop Epistemic Context Learning (ECL), a reasoning framework that conditions predictions on explicitly-built peer profiles from history.
  arXiv  Detail & Related papers  (2026-01-29T13:59:32Z)
• Why Authors and Maintainers Link (or Don't Link) Their PyPI Libraries to Code Repositories and Donation Platforms [83.16077040470975]
  Metadata of libraries on the Python Package Index (PyPI) plays a critical role in supporting the transparency, trust, and sustainability of open-source libraries.<n>This paper presents a large-scale empirical study combining two targeted surveys sent to 50,000 PyPI authors and maintainers.<n>We analyze more than 1,400 responses using large language model (LLM)-based topic modeling to uncover key motivations and barriers related to linking repositories and donation platforms.
  arXiv  Detail & Related papers  (2026-01-21T16:13:57Z)
• Transparent, Evaluable, and Accessible Data Agents: A Proof-of-Concept Framework [0.0]
  This article presents a modular, component-based architecture for developing and evaluating AI agents.<n>The system addresses core challenges in data accessibility by enabling non-technical users to interact with complex data warehouses.<n>A cornerstone of the design is its commitment to transparent decision-making, achieved through a multi-layered reasoning framework.
  arXiv  Detail & Related papers  (2025-09-28T23:54:41Z)
• CCE: Confidence-Consistency Evaluation for Time Series Anomaly Detection [56.302586730134806]
  We introduce Confidence-Consistency Evaluation (CCE), a novel evaluation metric.<n>CCE simultaneously measures prediction confidence and uncertainty consistency.<n>We also establish RankEval, a benchmark for comparing the ranking capabilities of various metrics.
  arXiv  Detail & Related papers  (2025-09-01T03:38:38Z)
• A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code [49.009041488527544]
  A.S.E is a repository-level evaluation benchmark for assessing the security of AI-generated code.<n>Current large language models (LLMs) still struggle with secure coding.<n>A larger reasoning budget does not necessarily lead to better code generation.
  arXiv  Detail & Related papers  (2025-08-25T15:11:11Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness [58.721012475577716]
  Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, prompting a surge in their practical applications. This paper introduces TrustScore, a framework based on the concept of Behavioral Consistency, which evaluates whether an LLMs response aligns with its intrinsic knowledge.
  arXiv  Detail & Related papers  (2024-02-19T21:12:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.