The Spectrum of SBOM Quality: From Useless to Fully Operationalized Cybersecurity Intelligence

To be blunt, a software bill of materials (SBOM) can range from a powerful tool for cyber supply chain risk management to, well, practically worthless.

This might come as a surprise to anyone generating SBOMs for the first time, or to anyone following the Executive Orders (EO) and Executive Mandates between May of 2021 and January of this year (EO 14028, mandate M-22-18, M-23-16, and EO 14144).?But the fact of the matter is this:

Most SBOMs are barely valid, few meet minimum government requirements, and almost none are useful.

Which is where this article comes in...

We’re about to review the four levels of SBOM quality:

1. bad, ugly, horrible, useless,
2. valid (but still useless),
3. one that meets minimum requirements (but is still useless), and
4. a fully operationalizable SBOM – the gold standard.?

Of course, a?fully operationalizable SBOM is what you’re after. That SBOM, in addition to meeting official mandates, will help you make sense of the threats and risks in your systems and applications. But before describing what that perfect SBOM looks like, let’s understand why we’re creating an SBOM in the first place (beyond, of course, meeting government requirements).

Not All SBOMs Serve the Same Purpose

Not all SBOMs serve the same purpose. The data they include depends heavily on the intended use case. Broadly speaking, SBOMs fall into two major categories:

1. Build Integrity and Reproducibility SBOMs, which focus on the software build process, ensuring that a package’s final binary matches its original build. These SBOMs capture details such as toolchain versions, compiler flags, and build attestations to help verify that a package has not been tampered with. A common example is the SBOMs generated by Yocto, which enable users to reconstruct a software package and confirm its integrity. While crucial for verifying software authenticity, these SBOMs provide little value in analyzing software composition and risk from a security perspective.
2. Software Composition and Supply Chain Security SBOMs, which focus on identifying the individual software components that make up the root package, making them useful for security analysis, compliance, and vulnerability management. The key to making these SBOMs “operationalizable” is ensuring that each component is accurately and uniquely identified. We think the Package URL (PURL) is the best identifier for this purpose, as it provides precise package metadata, allowing security tools to map dependencies and analyze risks effectively. Your comments are welcome.

For our purposes—software supply chain risk management—we focus on software composition SBOMs rather than build integrity SBOMs. The most valuable SBOMs for security analysis include precise package metadata, dependency relationships, and unique identifiers (such as PURLs), allowing organizations to conduct vulnerability scans, contributor risk analysis, and compliance checks with high accuracy.

But before we get to that, lets look at what a bad SBOM looks like (so we can do our best to avoid it).

A Terrible, Horrible, No-Good, Very Bad SBOM

We'll keep this short... A bad SBOM fails at its core purpose: transparency. It may be incomplete, outdated, or misleading. Common issues include:

• Missing or incorrect component listings
• Lack of versioning information
• Proprietary formats (i.e., not CycloneDX or SPDX) that hinder interoperability
• No update mechanism, leading to stale data
• Lack of trust indicators such as signatures or provenance details

This example illustrates a hand-generated text file masquerading as an SBOM with incomplete metadata. It lacks versioning information for dependencies, does not track relationships between components, and does not provide provenance details, making it difficult to use for transparency and basic tracking.

Several free tools can help validate an SBOM to ensure it meets format and completeness requirements. The SPDX Toolset provides validation for SPDX-formatted SBOMs, checking for schema correctness and completeness. CycloneDX CLI allows validation of CycloneDX SBOMs, ensuring they conform to the specification. Syft by Anchore can generate and validate SBOMs in multiple formats, including SPDX and CycloneDX. OSV-Scanner by Google helps check SBOMs for vulnerabilities by mapping dependencies to known security issues. SBOM Scorecard is another emerging tool that assesses the quality of an SBOM based on industry best practices. These tools help ensure that an SBOM is not only valid but also useful for security analysis.

Which brings us to the next best type of SBOM...?

A Valid SBOM

A valid SBOM conforms to the letter of the industry-recognized standard format (even if it is violating the spirit of those formats), making it technically sound, though not useful for cybersecurity. Typically, a valid SBOM:

• Lists software components, versions, and suppliers
• Uses a recognized format like SPDX, CycloneDX, or SWID
• Provides basic relationships between components
• Includes minimal metadata to support compliance efforts

Again, tools like SPDX Toolset, CycloneDX CLI, Syft by Anchore, OSV-Scanner by Google, and SBOM Scorecard are free tools that can, in some cases, be used to generate an SBOM and, in all cases, be used to validate an SBOM.?

Notice, neither of these example SBOMs share any components, dependencies, or other information that tells anybody anything about the software. And they certainly?don't give us enough information to perform any kind of cybersecurity analysis on. Yet, they're still considered valid.?

This is easy to fix though. Just demand a few minimum elements, and our SBOMs will be monumentally better (though still not yet ready for cyber operations).?

A Meets-Minimum-Requirements SBOM

Meeting government or industry requirements, the meets-minimum-requirements SBOM is legally or contractually compliant but still falls short of full security utility. The NTIA’s minimum elements, for example, require:

• Supplier Name
• Component Name
• Version of the Component
• Other Unique Identifiers
• Dependency Relationship
• Author of SBOM Data
• Timestamp

Other frameworks, like NIST’s SSDF (Secure Software Development Framework), add additional expectations but do not go beyond minimal compliance. And while these additional fields and requirements are great, we're looking for something specific. We're looking for a set of fields that allow us to extract relevant cybersecurity information.

We're looking for a fully operationalizable SBOM.?

A Fully Operationalizable SBOM

This SBOM is a strategic asset complete with specific data points that enable security and risk analysis. To make an SBOM fully operationalizable, it should be valid, it should contain minimum elements, and it should also contain the following:

• Package URLs (PURLs) – Provides precise package identification, enabling reliable sourcing and analysis.
• Cryptographic Hashes – Ensures integrity verification and malware detection.
• Software Dependency Graphs – Captures transitive dependencies for comprehensive risk assessment.
• Component Provenance – Tracks the origin of components to establish trust and detect supply chain risks.
• SBOM Signing and Attestations – Validates authenticity and ensures tamper resistance.
• Lifecycle Metadata – Includes versioning history, deprecation warnings, and maintenance status.
• Licensing Data – Specifies open-source and commercial license terms for compliance and risk assessment.
• Vulnerability Mapping – Links directly to CVEs and real-time security feeds for proactive threat detection.?
• Provenance and Build Transparency – Links to build attestations, cryptographic hashes, and SBOM signing for authenticity.

A fully operationalizable SBOM contains all the necessary fields to enable humans and automated tools to collect and analyze data relevant to the security of the components and dependencies in the SBOM so that our software supply chain isn't compromised.?By including these key data points, your SBOM becomes a security asset, enabling the following types of analysis:

• Contributor Analysis helps you measure the trustworthiness of maintainers and contributors. It allows you to identify foreign ownership, control, and influence (FOCI) concerns. It helps you identify developers who are contributing from State Department or DoD sanctioned/restricted entities. And it points out contributors who are associated with malicious or criminal activities. You can't trust the code if you can't trust the developers who wrote it.?
• Vulnerability Intelligence allows you to receive up-to-date vulnerability assessments as new vulnerabilities are discovered without having to rely on vulnerability data mapped directly in the SBOM, which may not be updated regularly.
• License and Compliance Checks help prevent legal risks, improper license use, and make you aware of legal obligations. This is especially important for components that may have declared a particular license but actually have alternate licenses or copyrighted information in the code itself.?
• Malware and Tampering Analysis like cryptographic integrity checks, malware scans, and code signing verification help keep known exploits out of your code.?
• Code Quality and Maintainability Scoring help you assesses technical debt, maintainability, and community activity in packages used by your software so that you can know in advance whether you may run into issues that prevent or deter you from maintaining your applications and systems in the future.?

It’s Time to Upgrade Your SBOM

As SBOM adoption accelerates, organizations must recognize that simply having an SBOM is not enough. A bad SBOM is useless. A valid SBOM?creates a false sense of security. A minimum-standards SBOM meets regulatory obligations to ensure compliance. But a fully operationalized SBOM provides actionable intelligence for proactive security.

Organizations looking to manage software supply chain risk effectively should aim for the highest possible tier.

As software supply chain attacks increase, the push for more detailed and security-centric SBOMs will grow. Government mandates, industry frameworks, and real-world security needs will likely drive future SBOMs toward deeper security integration, automation, and real-time threat intelligence. With that, the question will morph from “do you have an SBOM?” to “is your SBOM actually helping secure your software supply chain?”

If the answer to the question, "So you have an SBOM, now what?" is "Nothing, because my SBOM doesn't give me enough information to do anything else," it's time to upgrade.?

About Dark Sky Technology:

Dark Sky Technology is securing the world of software that powers our nations' most critical systems, devices, and applications by identifying malicious threats, untrustworthy contributors, risky code, and cyber attacks in open-source software. Our advanced analytics on open-source packages and their contributors protects the software supply chain and enables our customers to deploy secure, reliable, trusted software with confidence.

About Our Market (Who We Help):

We play in the open-source software security, software supply chain security, software supply chain risk management (CSCRM), and related markets. Most of our customers are aerospace and defense customers or government agencies. We aim to help these types of customers measure the riskiness and trustworthiness of using open-source software packages in their systems and software applications.

About our Product, Bulletproof Trust:

Our platform, Bulletproof Trust is a scalable software assurance and intelligence tool that measures the trustworthiness of open-source packages AND contributors. It scours various sources of online intelligence (OSINT) to analyze the health and status of and to identify malicious, criminal, or sanctioned contributors in an open-source package. Furthermore, it helps customers meeting the CISA Secure Software Development Attestation Form and NIST 800-161r1 requirements through software bill of materials (SBOM) generation and management.

Finally. Trust in Open Source.