Real-World Open Networking. Part 3 - Interoperability: Problems with Standards

In our last post we unpacked Interoperability, including Open Standards. Continuing this theme, we will look at how solution developers implement standards compliance and the problems that arise.

INTRODUCTION

Mandating that vendors (and internal systems) comply with Open Standards is a strategy used by organisations to drive interoperability. The assumption is that Open Standards compliant components will be interoperable.

In this post we examine the many reasons why that assumption does not always hold in real-world situations. This analysis will be from the Software perspective, since generally network equipment does a better job of component interoperability than software. This post will also cover general aspects of standards compliance in this post, and the specific aspects of API’s in the next post.

SOFTWARE IMPLEMENTATION & INTEROPERABILITY BASED ON STANDARDS

Whether the standard is “de jure” or “de facto”, there are three basic approaches to implementing software compliance with the standards:

• Reference implementation compatible
• Reference document compatible
• Architecture pattern or guideline compatible

REFERENCE IMPLEMENTATION COMPATIBLE

This approach consists of two parts:

• The standard is a controlling design input: i.e. compliance overrides other design inputs; and
• Validation against a “reference implementation” of the standard.

A “reference implementation” is a software component that is warranted to comply with the standard and is a known reference against which to validate a developed component. This should also include a set of standard test cases that verify compliance and / or highlight issues. 

Vendors often provide the test results as evidence and characterisation of the level of compliance. 

Benefits of this approach

This is the highest level of compliance possible against a standard. Two components that have been validated against the standard will be interoperable at the lowest common level to which they have both passed the test.

Problems with this approach

A reference implementation must exist and be available, however this is not always the case. The reference implementation must be independently developed and certified, often by the standards body themselves.

REFERENCE DOCUMENT COMPATIBLE

This approach is similar to the Reference Implementation approach. Firstly, the documented standard is a controlling design input. However the second part (validation against the standard) is both optional and highly variable. At the most basic level, compliance could be just that the vendor asserts component compliance with the standard. Alternatively, compliance may be validated by comparison between the developed component and the documentation, and there are many ways to do this at varying levels of accuracy and cost.

Benefits of this approach

The main benefits of this approach is that the design is driven to the standard, and at this level it is equivalent to the Reference implementation approach.

Problems with this approach

Validation without a reference implementation is highly manual and potentially subject to interpretation. This type of validation is very expensive which creates cost pressure for vendors to only partially validate, especially on repeat version upgrades and enhancements.

ARCHITECTURE PATTERN COMPATIBLE

In this case the standard is used as one input to the design, but not as the controlling input. The intent is not compliance but alignment. The product may use the same or similar underlying technologies as defined in the standards (e.g. REST interfaces or the same underlying data representation standards such as XML or JSON. The vendor may adopt a similar component architecture (e.g. microservices) to the standard.

Benefits of this approach

At best, this approach may provide a foundation for future compliance.

Problems with this approach

In general, the vendor is designing their product to be “not incompatible” with the standard, without taking on the cost of full compliance.

RATIONALE FOR VENDORS TO IMPLEMENT STANDARDS

Standards compliance is expensive to implement, regardless of the approach taken. So each vendor will take on their own approach, based on their own situations and context. A vendor may:

• Completely ignore the standards issue
• Deliberately, e.g. a start-up, whose early target customers don’t care.
• Accidentally, if they are unaware of the standards.
• Not see a competitive advantage in their marketplace: not so much as to justify the cost of standards implementation.
• Adopt a customisation approach: in other words, implement standardisation when required.
• Have full compliance in their roadmap for future implementation and simply want a foundation to build on.

PROBLEMS WITH COMPLIANCE

There are a wide range of implementations and the results are highly variable. The important thing to remember is that a claim of “standards compliance” can mean many things.

From a starting point of the intent to comply (or at least claim compliance), and using any of the strategies above, a vendor can be non-compliant in many ways:

• Partial implementation, e.g. a custom solution for one customer that is “productised”;
• Defects in implementation, including misinterpretation of the standard;
• Deliberate forking of the standard, including the implementation of superset functionality (“our solution is better than the standard”);
• The incompatibility of underlying or related components;
• Compliance with limited subsets of the standard, e.g. the most often used functions;
• Some vendors may misrepresent compliance based on tenuous connections: e.g. , a vendor might claim compatibility on the basis that their API’s are REST-based and nothing more.

CONCLUSION

Nothing can be assumed about standards compliance, other than that each vendor’s claims must be validated. The other part of this issue is Application Program Interfaces (API) interoperability. We will cover this in the next post. Stay tuned.