CryptoCurrency Security Standard (CCSS) - Audit and Pen-Testing Requirements

[Article Updated August 2023 for CCSS Version 8.1]

Marc Krisjanous is one of the first CCSS Auditors, assisted C4 in the development of their auditors program and is a member of the CCSS Steering Committee.

**** Free CCSS Implementation Guide! ****

Marc also co-authored the CCSS Implementation Guide for a Full System - click here to download - it's free!

In this series we will review each of the core Aspects in the CCSS and provide our interpretation for each of the Aspect's requirements and what possible evidence could provide assurance to the auditor that a requirement is in-place.?

In this article we will explore how an auditor could interpret the CCSS Aspect 2.01 Security Tests/ Audits.?

Aspect 2.01 Security Tests/ Audits addresses the importance of an independent (third-party) review of the in-scope people, process, and technology. The Aspect defines vulnerability scans, penetration tests and security audits. The Aspect’s objective defined within the CCSS?is provided below.?

"This aspect covers third-party reviews of the security systems, technical controls, and policies that protect the information system from all forms of risk as well as vulnerability and penetration tests designed to identify paths around existing controls. Regardless of the technical skills, knowledge, and experience of staff who build and maintain an information system, it has been proven that third-person reviews often identify risks and control deficiencies that were either overlooked or underestimated by staff. For the same reasons that development companies require different people to test a product from those who write it, different people than those who implement a cryptocurrency system should assess its security. Third parties provide a different viewpoint and are independent of the technical controls and can be objective without risk of retaliation."

Vulnerability Scanning and Pen-tests - there is a difference?

It's important to note the key differences between a vulnerability scan and a penetration test.

A vulnerability scan is a process where?systems are scanned for vulnerabilities. The scan can be in-depth in which the scanner has login credentials to servers/devices and systems - scanning from the operating system level and up the software stack. This type of in-depth scan is often referred to as an “internal vulnerability scan” as the scanner is scanning internal network servers, devices, and systems.

Another type of scan is an “external vulnerability scan” which just scans the publicly accessible interfaces of a system, for example a web application's public web pages and web server. The external scanner may also have login credentials and log into the web application to scan more web pages.?

A penetration test or "pen-test" comprises firstly of a vulnerability scan #1 to identify vulnerabilities in the target systems. The vulnerabilities identified are reviewed by the pen-testers who then decide what vulnerabilities to attack to gain unauthorized access to the target systems. If the attack is successful and the pen-tester gains unauthorized access to the target systems then the pen-tester, if instructed by the client, will attempt to see how much further they can get into the systems by targeting other vulnerabilities or weaknesses in the system that could not have been exploited initially.??

Code Review/Audit?

The review of code for vulnerabilities can be conducted using tools that analyze the code or are undertaken by a code reviewer who is skilled in the coding language. Often both approaches are utilized and not only are industry best practices referenced as a guide during the review process, but the assessed entities coding standards are referenced to ensure the standards have been adhered to. CCSS also requires that the review/audit include a review of the cryptographic libraries used to ensure they have been implemented correctly.??

CCSS Levels?

CCSS provides three levels of compliance - Level 1 being the base level of implementing CCSS requirements up to Level 3 being the most in-depth implementation of CCSS requirements. We shall review each compliance level and provide our thoughts on what evidence an auditor should seek to provide assurance that the requirements are in-place.

CCSS Level 1 Requirements?

CCSS Level 1 requirements for this Aspect are provided below.??

Requirement: 2.01.1.1 - A developer who is knowledgeable about digital asset security has assisted in the design and implementation of the information system and documentation of an internal assessment exists.

There are two parts to this requirement:

1. A developer, who has the required skills, assisted in the design and implementation of the information system and,
2. Documentation of an internal assessment exists.

The first part of the requirement requires that a developer, skilled in the cryptocurrency protocol being implemented, assists in the design and implementation of the information system using the cryptocurrency. Evidence that provides assurance this requirement is in-place could be:?

1. The developer(s) roles and responsibilities for the project are documented and approved by senior management who have accountability for the project.?
2. Evidence of the developer’s skill and experience in the cryptocurrency has been reviewed by the relevant project members or other resources to ensure the developer is skilled enough for the role. Evidence provided could be certifications for the development, design and or implementation of the cryptocurrency or other relevant technologies. Other company references for the developer could be included as well.
3. We would also recommend that the auditor seeks some form of evidence that the developer has a good understanding of secure coding techniques - preferably the developer has undertaken secure coding training relevant to the coding language and platform. The developer may have the required skill for the cryptocurrency being used but if they do not know and apply the basics of secure coding techniques then this is a major risk to the project. For example, hard coding a private key into the source code is not a good idea but it does happen.?

The second part of the requirement requires that documentation of an internal assessment exists. It is generally considered that an internal assessment is undertaken by internal personnel to ensure that the information system (people, process and technology components) meet the requirements of an external standard. For example, the Governance, Risk and Compliance (GRC) team of an organization may conduct an internal assessment every quarter to ensure that they remain compliant with the requirements of a standard or certification program such PCI DSS. This ensures that when an audit, undertaken by external third-party auditors, is performed when the current certification expires, that the organization does not fail to meet the requirements.

The CCSSA should review the policy and procedures that define when an internal assessment is required and should include the events that would trigger an internal assessment such as a significant change to the environment and/or based on a date schedule (e.g. quarterly reviews).

The CCSSA should review all outputs of the internal assessment process conducted within the assessed period such as assessment reports and documented remediation activities to ensure they meet the requirements of the policy and ensure that the assessment was conducted by qualified and skilled personnel.

CCSS Level 2 Requirements?

CCSS Level 2 requirements for this Aspect are provided below.?

Requirement: 2.01.1.2 - A regular security assessment that includes vulnerability and penetration testing has been completed by an independent qualified third-party. Documentation shows that all concerns raised by the assessment have been evaluated for risk and addressed by the organization.??

As mentioned above, there is a key difference between a vulnerability scan and a penetration test. To put it another way - a vulnerability scan will detect "holes" in the system. Where a penetration test will detect the "holes" and use one or more holes to gain access into the systems behind and see what other vulnerabilities are around. The goal of most penetration tests is to see if the pen-tester can exploit the vulnerabilities to a level where they have complete control of all systems in-scope.?

Level 2 requirements state that regular security assessments are conducted which include vulnerability scanning and pen-testing. Breaking this requirement down into more manageable pieces for audit we have identified the following audit considerations:??

1. Level 2 requires a regular security assessment that includes a vulnerability scan and pen-test. The CCSS Glossary defines "Regularly" as being annual. Conducting a security assessment only once a year may not be enough to ensure the information systems are secure especially after major changes to the systems. It is our opinion that the assessed entity should define the frequency based on several factors including: (1) the relevant risks identified by the assessed entity for their cryptocurrency products and services, (2) the amount of change and the complexity of the change for systems that provide cryptocurrency functions. For example, changing software coding language and having to rewrite many components should trigger a security assessment regardless if a security assessment was completed a month before, (3) regulatory or statutory requirements. ?
2. Depending on factors such as the scope of the testing, the skill of the pen-testers, time given to test the in-scope environment, and the configuration and hardening applied to the systems - penetration testing and vulnerability scan findings may be considerable, requiring a great deal of time to remediate. The findings might be so severe as to cause the entire system to be removed from production until fixed.?
3. Level 2 requires that the auditing and testing?tasks are conducted by a third-party entity. There is currently no definition provided by the CCSS Committee for "third party entity" #2 so let us refer to the International Organization for Standardization (ISO) terms of reference database. ISO provides a couple of definitions for "third party" depending on the standard the term is applied to.?A definition that is applicable to our needs is "person or body that is recognized as being independent of the parties involved with respect to the issues in question". #3 The key word in this definition is "independent". This definition is still too vague for most audits as "independent" could be defined as a person or team who is employed by the same organization that is building and implementing the system but not directly involved with that project. Is that interpretation acceptable for you as an auditor or would you require an entity outside of the in-scope organization? As an auditor you may have to discuss with the assessed entity and reach an agreement on what "third party entity" means. Not to go on, but another consideration is what type of penetration testing was conducted? Was it Internal? External? Segmentation? Blackbox? Whitebox? Grey Box?
4. On the subject of code auditing, vulnerability scanning and penetration testing - CCSS does not provide any guidance as to the professionalism or certification requirements of the third party who will conduct these tasks. As an example, the assessed entities CEO's 20-year-old son who dabbles in "hacking" using Kali Linux every night in his room, in our opinion, would not be recognized by Confide's auditors as being a professional penetration tester who holds formal qualifications in penetration testing, works for a company that is Crest #4 certified (for example) and uses an industry recognized standards-based approach for penetration testing such as NIST SP800-115. #5
5. There is also a requirement for remediation evidence: "Documentation shows that all concerns raised by the assessment have been evaluated for risk and addressed by the organization.". Firstly, we will assume that by "assessment" CCSS means the findings from the vulnerability scan or the penetration test. The findings should be provided as official documentation to the auditor, meaning the report authored by the third-party conducting the audit, with all findings listed. The auditor should ensure that the report has not been edited to hide any findings from the audit. As part of the evidence, a remediation list should also be provided to the auditor which outlines the assessed entities assessment of each finding as to whether they believe the finding to be a false positive or a valid finding, what is the remediation to address the issue, the date the finding will be addressed (e.g fixed) and how will the remediation be checked to show that each finding has been successfully addressed. CCSS does not define if another assessment in full or in-part, vulnerability scan or penetration test needs to be conducted once all accepted findings have been addressed to confirm all findings have been sufficiently remediated. This leaves the auditor with the challenge as to how he/she can confirm the remediation has been completed successfully.?

CCSS Level 3 Requirements?

CCSS Level 3 requirements for this aspect is provided below.?

Requirement: 2.01.1.3 - A regular security audit at a level similar to SOC2, ISAE3402, or ISO-27001, that includes vulnerability, penetration testing, and code audit (if applicable) has been completed by an independent qualified third-party. Documentation shows that all concerns raised by the audit have been evaluated for risk, addressed by the organization, and known vulnerabilities have been removed from the system. Ongoing audits are scheduled on a (minimum) yearly basis.

The Level 3 requirements provide a subtle change in the wording from Level 2. Note that for Level 2 compliance a security assessment is required but for Level 3 compliance an audit is required.

What's the difference between an assessment and an audit?

In our opinion, an audit is a process conducted by a third-party who is outside of the organization #6 that reviews evidence collected from the entity being audited and verifies their compliance to a standard or policy. An assessment process reviews the evidence provided and reaches a conclusion regarding the state of the entity at that point in time, in readiness for compliance to a standard or policy.

For example, you may have heard of a "readiness assessment for audit". This is where a person(s) internal or external to the organization, can review the environment that will be audited and determine what will be compliant and what will not. Then a remediation plan is generally created so the organization can implement changes to reach compliance.

CCSS for Level 3 provides a required frequency for audits which is at a minimum on a yearly basis. It should be noted that CCSS does not require an audit to be conducted if there are any significant changes to the CCSS in-scope environment.

Summary?

In this post we explored Aspect 2.01 Security Tests/ Audits with our auditors hat on to determine how an auditor could approach the evidence gathering required so that an opinion could be reached as to if the assessed entity is compliant to Level 1, Level 2 or Level 3 for this Aspect.

We discovered that many of the Aspect's requirements required a reasonable amount of interpretation which adds risk that the assessed entity may reach a different interpretation than the auditor which is never a good situation to be in for either party.

Ultimately, it is the auditor who risks their standing, auditing license and auditing career by signing a certification of compliance for the assessed entity. The auditor must be confident that the requirements have been meet and that the standard provides enough detail and clarity that both the assessed entity and the auditor can reach the same conclusion as to what evidence is required to prove that the requirement is in-place.?

CCSS is still a “young” information security management systems standard however, the standards objectives are very clear, and we believe CCSS will improve and mature overtime and be a highly respected standard to be compliant in.?

#1 pen-tests almost always start with a vulnerability scan to detect vulnerabilities. For some penetration test engagements, the assessed entity may provide a list of vulnerabilities to attack, or the penetration testers are aware of the vulnerabilities inherent to the systems already.?

#2 https://cryptoconsortium.org/cryptocurrency-security-standard-auditor-ccssa-glossary/?

#3 https://www.iso.org/obp/ui#search?

#4 https://www.crest-approved.org/certification-careers/crest-certifications/?

#5 https://csrc.nist.gov/publications/detail/sp/800-115/final?

#6 There are audits that are undertaken by internal auditors, called an internal audit, but CCSS requires a third-party audit, which we have defined as completely independent of the assessed entity.

More Research

An introduction to CCSS - Confide.co.nz

What is a CCSSA? - Confide.co.nz

PCI DSS & Cryptocurrency Security Standard (CCSS) – A Marriage Made for Crypto Security

Confide.co.nz