New Year’s Resolution: More Assurance, Less Seat of the Pants

Using Assurance Cases to Demonstrate Systems Are Trustworthy Secure

With today’s cutting-edge computing technologies driving all types of systems including Information Technology (IT) systems, Operational Technology (OT) systems, and cyber-physical systems, how do you know if your system is actually trustworthy and secure? The term trustworthiness in this context means worthy of being trusted to fulfill whatever critical requirements may be needed for a particular system, component, subsystem, network, application, mission, enterprise, or other entity [1]. The concept of trustworthiness is grounded on something known as an “assurance case.” Assurance cases provide reasoned, auditable artifacts that support the contention that a claim or set of claims is satisfied, including systematic argumentation and underlying evidence and explicit assumptions that support the claims [2].

Evaluation has been the traditional means of obtaining assurance for systems to determine if they are trustworthy and secure. Evaluation techniques include such activities as:

• Analysis and checking of processes and procedures
• Checking that processes and procedures are being applied
• Analysis of the completeness, consistency, and accuracy of requirements
• Analysis of the correspondence between system and system component design representations
• Analysis of the system and system component design representation against the requirements
• Analysis of the usability of the system
• Verification of proofs
• Analysis of guidance documents
• Analysis of functional tests and the results provided
• Independent functional testing
• Vulnerability analysis (including flaw hypothesis)
• Penetration testing

The above activities produce “assurance evidence” that can be used to verify the security and trustworthiness of systems and system components. Such assurance evidence can be generated from any type of system development life cycle process including the state-of-the-practice Agile and DevOps processes.

The problem with today’s security mindset is that we are not investing in comprehensive assurance cases. Assurance cases can bring together the many security-related activities carried out during the system development life cycle including threat modeling, the development of security requirements and security architectures, formal modeling, design analyses, code reviews, inspections, and security testing and evaluation. Assurance cases can be used to answer some very simple questions for end users:

• What threats do you care about with respect to your organizational assets that require protection?
• How much loss are you willing to accept from an organizational perspective?
• How inherently safe and secure is your system from loss and its consequences?
• What security measures (i.e., safeguards) have been selected to protect your assets?
• What evidence demonstrates that the selected security measures are correct and effective?

Evidence generated during the system development life cycle is essential to building assurance cases for systems and system components being deployed in the critical infrastructure. Assurance cases can turn security into something that is concrete, measurable, and sharable. They can be built iteratively as conditions change and developers react to new threats or previous threats that have not been addressed with adequate safeguards.

Assurance evidence can be shared with end users through security labeling programs [3] that reflect the assurance arguments made in the assurance cases. The assurance evidence captured during the system development life cycle provides the analytical depth, rigor, visibility, and transparency necessary for end users to make credible decisions on whether or not to trust the systems and systems components in these critical operational environments.

Unless and until we bring back the concept of assurance and assurance evidence into the mainstream of systems development activities, we will continue to make decisions that feel good but provide little value in determining if systems and their constituent components are demonstrably trustworthy. Systems with “engineered security” and measurable assurance are the gateway to trustworthy secure systems and protecting the Nation’s critical assets.

NIST SP 800-160, Volume 1, Revision 1 begins to address this problem in the upcoming update to be released in January 2022. This initial public draft includes a dedicated section on the trustworthiness and assurance of systems and system components. Focusing on assurance-related activities (including assurance cases) throughout the system life cycle can produce trustworthy secure systems, increase transparency and visibility for end users, and reduce costly post-deployment testing that produces little, if any, return on investment.

Building and delivering assurance is the way to drive the “culture of security.” Let’s make 2022 the year of “security assurance.”?

A special note of thanks to Mark Winstead and Jeff Williams, long-time cybersecurity and SSE colleagues, who graciously reviewed and provided sage advice for this article.

[1]??P. Neumann, “Principled Assuredly Trustworthy Composable Architectures.”

[2]??ISO/IEC 15026-2:2011, Systems and software engineering — Systems and software assurance — Part 2: Assurance case.

[3]??Executive Order (EO) 14028, Improving the Nation’s Cybersecurity.