A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM

• URL: http://arxiv.org/abs/2601.05622v1
• Date: Fri, 09 Jan 2026 08:26:05 GMT
• Title: A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM
• Authors: Chengjie Wang, Jingzheng Wu, Hao Lyu, Xiang Ling, Tianyue Luo, Yanjun Wu, Chen Zhao,
• Abstract summary: A Software Bill of Materials (SBOM) is a machine-readable artifact that organizes software information.<n>Following standards, organizations have developed tools for generating and utilizing SBOMs.<n>This paper presents the first large-scale, two-stage empirical analysis of the adherence gap, using our automated evaluation framework, SAP.
• Score: 54.38424417079265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A Software Bill of Materials (SBOM) is a machine-readable artifact that systematically organizes software information, enhancing supply chain transparency and security. To facilitate the exchange and utilization of SBOMs, organizations such as the Linux Foundation and OWASP have proposed SBOM standards. Following standards, organizations have developed tools for generating and utilizing SBOMs. However, limited research has examined the adherence of these SBOM tools to standard specifications, a gap that could lead to compliance failures and disruptions in SBOM utilization. This paper presents the first large-scale, two-stage empirical analysis of the adherence gap, using our automated evaluation framework, SAP. The evaluation, comprising a baseline evaluation and a one-year longitudinal follow-up, covers 55,444 SBOMs generated by six SBOM tools from 3,287 real-world repositories. Our analysis reveals persistent, fundamental limitations in current SBOM tools: (1) inadequate compliance support with policy requirements; (2) poor tool consistencies, including inter-tool consistency rates as low as 7.84% to 12.77% for package detection across languages, and significant longitudinal inconsistency, where tools show low consistency with their own prior versions; and (3) mediocre to poor accuracy for detailed software information, e.g., accuracy of package licenses below 20%. We analyze the root causes of these gaps and provide practical solutions. All the code, replication docker image, evaluation results are open sourced at [GitHub](https://github.com/dw763j/SAP) and [Zenodo](https://doi.org/10.5281/zenodo.14998624) for further researches.

Related papers

• UniBOM -- A Unified SBOM Analysis and Visualisation Tool for IoT Systems and Beyond [0.23332469289621785]
  This paper introduces UniBOM, an advanced tool for Software Bill of Materials generation, analysis, and visualisation.<n>UniBOM integrates binary, vulnerability, and source code analysis, enabling fine-grained vulnerability detection and risk management.<n>Key features include historical tracking, AI-based classification by severity and memory safety, and support for non-package-managed C/C++ dependencies.
  arXiv  Detail & Related papers  (2025-11-27T11:50:58Z)
• InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents [60.89180545430896]
  InfoMosaic-Bench is the first benchmark dedicated to multi-source information seeking in tool-augmented agents.<n>It requires agents to combine general-purpose search with domain-specific tools.<n>This design guarantees both reliability and non-triviality.
  arXiv  Detail & Related papers  (2025-10-02T17:48:03Z)
• Policy-driven Software Bill of Materials on GitHub: An Empirical Study [14.398115591070727]
  The Software Bill of Materials (SBOM) is a machine-readable list of all the software dependencies included in a software.<n>Despite mandates from governments to use SBOM, research on this artifact is still in its early stages.
  arXiv  Detail & Related papers  (2025-09-01T08:45:39Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Search [51.91311158085973]
  multimodal misinformation often arises from mixed forgery sources, requiring dynamic reasoning and adaptive verification.<n>We propose T2Agent, a novel misinformation detection agent that incorporates a toolkit with Monte Carlo Tree Search.<n>Extensive experiments show that T2Agent consistently outperforms existing baselines on challenging mixed-source multimodal misinformation benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T09:50:55Z)
• A Dataset of Software Bill of Materials for Evaluating SBOM Consumption Tools [6.081142345739704]
  A Software Bill of Materials (SBOM) is a list of components used in software.<n> Numerous tools support software dependency management through SBOMs.<n>There is no publicly available dataset specifically designed for this purpose.<n>We present a dataset of SBOMs generated from real-world Java projects.
  arXiv  Detail & Related papers  (2025-04-09T13:35:02Z)
• Vexed by VEX tools: Consistency evaluation of container vulnerability scanners [0.0]
  This paper presents a study that analyzed state-of-the-art vulnerability scanning tools applied to containers.<n>We have focused the work on tools following the Vulnerability Exploitability eXchange (VEX) format.
  arXiv  Detail & Related papers  (2025-03-18T16:22:43Z)
• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use.<n>MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space.<n>MeCo is fine-tuning-free and incurs minimal cost.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• Supply Chain Insecurity: The Lack of Integrity Protection in SBOM Solutions [0.0]
  The Software Bill of Materials (SBOM) is paramount in ensuring software supply chain security.<n>Under the Executive Order issued by President Biden, the adoption of the SBOM has become obligatory within the United States.<n>We present an in-depth and systematic investigation of the trust that can be put into the output of SBOMs.
  arXiv  Detail & Related papers  (2024-12-06T15:52:12Z)
• The Impact of SBOM Generators on Vulnerability Assessment in Python: A Comparison and a Novel Approach [56.4040698609393]
  Software Bill of Materials (SBOM) has been promoted as a tool to increase transparency and verifiability in software composition. Current SBOM generation tools often suffer from inaccuracies in identifying components and dependencies. We propose PIP-sbom, a novel pip-inspired solution that addresses their shortcomings.
  arXiv  Detail & Related papers  (2024-09-10T10:12:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.