Kubernetes & The Quest for Orchestration

Kubernetes adoption is growing rapidly, becoming the de-facto standard platform for running cloud-native applications. Having said that, more than half (52%) of companies are not widely adopting Kubernetes within their organizations due to a lack of expertise. This finding highlights the operational complexity of Kubernetes deployment and management and the acute need for management solutions which was cited as the second largest adoption challenge by half (50%) of the companies according to a recent survey.

Andrew van Rooyen post: Learning the hard way: Microservices provides a more specific insight into the specific challenges involved in the Cloud Native transformation journey which bring more organization to realise that while Kuberntes is the de-facto standard for running Cloud Native applications it is not a “one size fits all” and there would be many cases in which the investment and the complexity involved in running Cloud Native doesn’t make sense.

The investment cycle of Cloud Native Transformation Journey 

In this article, I wanted to elaborate how Cloudify addresses those challenges by providing a layer of automation that integrate Kubernetes with the existing enterprise environment and applications and thus smooth the transformation and skillset complexity. I will also describe how Cloudify simplifies the third biggest challenges - multi-cluster management (38%) in a manner that supports the management of Kubernetes and non-Kubernetes clusters from core to edge.

I will conclude by revealing some new innovations that we’re currently working on, which includes the ability to accelerate the performance of I/O intensive cloud native services through specific hardware acceleration optimization that enable the running of container services at wire performance.

Recap - Cloudify vs Kubernetes ?

Those that are new to orchestration would tend to see any orchestration as part of one big  automation category, and therefore the question of Cloudify vs Kubernetes still comes up from time to time. 

I started to address this question in one of my earlier posts on the subject - Why Do I Need TOSCA If I’m Using Kubernetes? Part II Of II where I pointed out the natural evolution process of new technologies like Kuberentes, and why you would need a layer like TOSCA and Cloudify to provide an end to end automation between Kubernetes with non Kubernetes services.

We are now at a point in time where it becomes clearer there ain't going to be one orchestration  to win them all. The orchestration world evolves into a world of many domain specific orchestrations some will be based on Kubernetes and some not. An example for that is the Cloud infrastructure orchestration such as AWS Cloud Formation and Azure ARM. Network and Edge orchestration is another good example so is Configuration management orchestration such as Ansible.

We are also seeing a movement toward new domain specific platforms that rely on Kubernetes as a substrate and expose their own domain specific interface on top. Many of the new edge platform such as Akraino or StarlinX are a good example on this regard.

In parallel to that we are also moving into a world in which will be relying more and more on services that are managed by the cloud providers as well as third party services for handling things like analytics and data processing , monitoring etc .

All of this makes the need for a multi domain service orchestration that can handle the end to end automation across those different domains even more broad than I originally thought, and the number of use cases that leads to such a need is growing rapidly. 

Cloudify - Multi Domain Services Orchestration for Kubernetes

Multi domain orchestration is not a new thing in Cloudify. In the earlier version of Cloudify we supported a simple set of domain mostly centered around multi-cloud infrastructure domains such as OpenStack, Vmware, AWS etc.

In the new releases of Cloudify we are moving up the stack to address more modern domains. Domains that often comes with their own orchestration and their own service modeling language. Kubernetes takes a central place as one of the key domains through the integration with Kubernetes service API. This integration allows users to define more complex microservice graphs alongside non-Kubernetes services under a common declarative (a.k.a  intent based) automation scheme.

In addition to that we added support for additional domain specific orchestration which include support for infrastructure domains orchestration such as AWS Cloud Formation and Azure ARM and Terraform as well as configuration management domain such as Ansible 

The diagram below shows all the domains that are currently supported by Cloudify in this new architecture. Each domain comes with its own set of plugins that enable the mapping of the domain specific API as a set of TOSCA objects (nodes). Having each domain available as a resource is only a first step to allow multi-domain service orchestration.

The next phase is to extend the modeling language in away that will support service modeling of the relationship semantics between those domains. Such a relationship will enable to define which specific properties need to be exposed between domains, how lifecycle operation will be mapped in the case of a dependency graph between multiple independent services. How workflow and properties can be overridden , how event of one service can trigger an event on another service etc.

This new modeling enhancement is referred to as Service Composition and is available as part of the new Cloudify 5.x release.

The combination of the new service-modeling and service plugins allows users to model an end to end service simply by pointing to the relevant object of the specific domain. Users can add an overlay blueprint that will map the specific relationship and dependencies of a specific deployment without having to modify the underlying service component. This allows users to pass context information , define service dependencies and workflows that work across all those entities.

Kubernetes users could now describe and automate an end to end service that is comprised out of Kubernetes microservices as well as third party services, infrastructure services, network services virutal (VNF) of physical (PNF)  under a common automation scheme.

To better illustrate how all this works we came up with a reference architecture that demonstrates how to use Cloudify to address multi-cloud and hybrid stack deployment.. In this reference architecture we demonstrate how we can put together services running as a cloud-native service such as wordpress and connect them with other services that run on top of traditional virtual machines in a multi-cloud environment as described in the diagram below:

Example 1: Using the enhanced service  modeling to automate a multi-cloud application

We also created a live demonstration that walks through the details of how all this works. This demo shows many of the core capabilities that need to take part in this sort of multi-cloud orchestration which includes network orchestration, security, configuration, scaling etc. Here is the full list:

• Multiple management networks (Multi-cloud)
• Traditional VMs, and Kubernetes orchestration (Hybrid-cloud & stack)
• Multi-tenancy
• Composite micro services orchestration 

 (via inter blueprint orchestration)

• Global and tenant secrets
• Scaling
• Coordinated multi-cloud deployment
• Coordinated hybrid cloud deployment
• Multi-blueprint composition
• Kubernetes cluster deployment
• Templatized Kubernetes resource deployment
• Openstack compute and network orchestration
• AWS compute and network orchestration
• Cross-template/cross-cloud scaling
• MariaDB, HAProxy, Drupal, and WordPress orchestration
• Service Chaining

Beyond Helm

Helm is a popular packaging platform for Kubernetes services. As such it provides a rich set of Helm Charts that support many services. Cloudify comes with Helm Plugin that allow users to deploy Helm based Kubernetes services in the same native Kubernetes services.

Having said that Helm is currently positioned to address more simple service use cases and still has many limitations in addressing more advanced services as described in this post: Think twice before using Helm.

The combination of Cloudify as a layer that can integrate, but at the same time be independent of Helm, allows Kubernetes users to leverage the rich set of Helm charts but at the same time interact directly with the native Kubernetes service templates to address more complex service scenarios.

Multi Site Cluster and Edge Orchestration 

Multi-cluster management was ranked as the most important capability for cluster management (34%) according to the survey.

Cloudify can provision each Kuberentes cluster as a separate service deployment and thus provide an easy way to manage many instances of Kubernetes clusters using the same management console.

Similarly Cloudify can point to an existing cluster of Kuberentes such as the managed Kubernetes cluster by Google or Azure or AWS and make it part of its cluster pool.

In addition to that, Cloudify provides a means for deploying a hybrid stack of Kubernetes and non Kubernetes clusters using the same automation blueprint.

Cloudify Spire provides the ability to manage Kubernetes clusters on the edge and at global scale through a federated management architecture.

This opens up the option to address higher order orchestration needs such as placement policies based on performance , location between those clusters as well as managing the availability of services between multiple clusters zones etc. 

The upcoming Cloudify Spire release provides a multi-site management that includes a location based management UI and criteria based matching to allow more flexible placement policies. You can learn more about Cloudify Spire here.

Making Cloudify Cloud Native 

In the previous section we covered the ability of Cloudify to orchestrate Kubernetes microservices on Kubernetes.

In addition to that we've been working to make Cloudify itself a first class citizen within the Kubernetes ecosystem by making Cloudify itself Cloud Native. This includes re-factoring of Cloudify core services into a smaller set of micro-services that are packaged as containers and can be deployed as a Kubernetes service or a Docker application outside of Kubernetes.

Making Cloudify First Class Citizen within Public Cloud

Cloudify was originally built to target private cloud deployments such as OpenStack. As such we couldn’t assume the availability of services such as Data Base as a service , Messaging as a service, object storage as well as load-balancer as a service being available in each infrastructure. As a result we had to package those services as part of each Cloudify deployment to make Cloudify portable across all possible private and public cloud environment.

 As we move to public cloud this assumption can be changed. AWS, Azure as well as Google support all those services as part of their infrastructure. This allows us to rely on the built-in public cloud services when running Cloudify on public cloud and therefore simplify dramatically the way we run Cloudify on those environments.

The first step on this regard was to enable integration of Cloudify with built-in data-base as a service on Azure and AWS.

The next phase would be to enable the rest of Cloudify services in similar fashion.

Enabling continues upgrade

The move to cloud-native architecture as well as reducing the number of moving parts per Cloudify deployment  by relying more on the Cloud infrastructure services enabled us to move to an active active cluster management as well as handle continues upgrade process between releases which simplifies the management of the cloudify manager significantly. Simplifying the cloudify manager maintenance becomes even more important as we move to federated management  which is comprised of many micro orchestrations. For example It enable us to upgrade an entire cluster of cloudify-managers in one go or perform any other maintenance task in batch mode across an entire cluster.

Use Cases

Cloud Native SD-WAN

There is a growing number of network services such as firewall , routers etc that are available today as cloud-native services.

This opens up a new way to deliver cost effective edge use cases such as SD-WAN and uCPE.

The diagram below shows how Cloudify can leverage open source cloud native services to deliver an open SD-WAN solution. This includes the remote management of the uCPE, as well as the life cycle management and service chaining of the VNF’s within that uCPE.

Supporting I/O intensive Cloud Native Services (Coming Soon..)

The default setup of Kubernetes was designed to support standard IT workload. Network services as well as data processing services tend to be latency and I/O sensitive.

To support this kind of services on top of Kubernetes we've integrated a special set of hardware acceleration capabilities which leverages the Intel EPA. Such capabilities includes features like QAT which enable SSL offloading as well as DPDK and SR-IOV.

Cloudify provides a way to run a Kubernetes cluster that is optimized to use this sort of hardware acceleration capability. 

In addition to that Cloudify provides placement policies that enable dynamic placement of services into the relevant Kubernetes cluster based on their performance characteristics using the Cloudify Spire multi-cluster management.

You can read more about this on the following post:  Workloads (Pods, VMs) placement on distributed Kubernetes and non-Kubernetes clusters utilizing Enhanced Platform Awareness (EPA) from Intel.

Managing ONAP Kuberentes and Big Data Clusters 

The DCAE project within ONAP serves as the data collection and analytics platform that is responsible for collecting all the data from the network services that are managed by ONAP.

The EOM and DCAE project in ONAP is using Cloudify to manage their environment which is comprised out of Kubernetes cluster as well as many other non Kubernetes services such as database services, big-data clusters etc.

Final notes

Kubernetes has evolved very rapidly to the point where it became the industry standard cloud native application platform. Having said that we are still seeing that many organizations are struggling in their journey to transform themselves into this new model from their existing environment and practices. This complexity leads many organizations to own a smaller part of their current stack and rely more on third party solutions that are self managed. In addition to that we expect that many of the legacy services will not go through this transformation. This leads to an environment where our system will be built out of a combination of new services built as Cloud Native services on top of Kubernetes that will need to interface with third party and legacy services that lives outside of their kubernetes cluster.

Delivering an end to end automation in such an environment requires an automation platform that can integrate with all those different domains. This has been our Cloudify approach to Kubernetes, and we recently extended our approach to support a wider range of domains that includes Infrastructure orchestration domains, third party services through our REST and Ansible plugins. We also added  a new multi-cluster management , as well as multi-site edge use cases.

In addition to that were seeing the expansion of cloud data centers to the edge as noted by Ajay Gulati in his post on the  The Era of Cloud Managed Infrastructure.

  “The big difference is how the infrastructure is being delivered and managed at the edge. That is going to be the main disruption for many existing vendors in this space, who are still following the older delivery, support and management models..”

This opened the door for us to move up the stack and address higher order orchestration needs such as placement policies that deals with match making between given workload and the cluster that fits best to serve this workfload. 

We are now working in close collaboration with Intel to add more performance acceleration capabilities to the Kubernetes cluster to support more I/O and latency sensitive services.

Stay Tuned!

References

• Cloudify New Multi Cloud Kubernetes Demo (Video)
• Workloads (Pods, VMs) placement on distributed Kubernetes and non-Kubernetes clusters utilizing Enhanced Platform Awareness (EPA) from Intel.
• New Kubernetes Survey
• Cloudify Spire Overview
• Why Do I Need * If I’m Using Kubernetes? Part I Of II.
• Why Do I Need TOSCA If I’m Using Kubernetes? Part II Of II 
• EOM and DCAE project in ONAP
• Multi-cloud and hybrid stack deployment.
• Cloudify Integration with Kubernetes service API
• AWS Outposts, GKE On-Prem, ZeroStack: The Era of Cloud Managed Infrastructure
• Learning the hard way: Microservices
• Think twice before using Helm.
• Kubernetes war stories