Solving the Federated API Management Problem

Author Biographies:?

• Wendy Cameron: https://www.dhirubhai.net/in/wendymcameron/ - Wendy is an IT Professional with 25 years of experience across New Zealand, Australia and the United Kingdom typically in large tier 1 or tier 2 enterprises with extensive background in integration and the API ecosystem.
• Josh Twist: https://www.dhirubhai.net/in/joshtwist/ - Josh Twist Co-founder & CEO at Zuplo with a track record of kick-starting zero-to-one projects and turning around broken-to-one opportunities. "UK Developer of the Year" award winner combining deep technical expertise with a passion for big-picture product strategy and a habit of finding tactics to unlock growth and unblock teams. Loves inspiring a team to climb the next big mountain.

Management Problem

Productivity, efficiency, and innovation are the keystones of any enterprise API program. This has never been more critical, with the paradigm shift caused by Generative AI - APIs are how AI gets real work done. The majority of AI processes cannot be developed locally in an isolated environment and can only be developed in Cloud using the full fledged toolkits offered by the major Cloud providers.

Every enterprise needs an API strategy to support their AI strategy.

Larger organizations with federated management and operations are facing new challenges as their API management estate grows. Large organizations might have hundreds of teams working on APIs, and thousands API endpoints. At some point, it becomes imperative for an organization to drive for consistency and governance across this landscape, or rather, hellscape: imagine the sprawling mess when authentication, protection, authorization, and policy decisions are distributed to independent teams.

Federated API Management Use Cases

Some federated API management use cases are:

• Enabling teams and organizations to inherit and leverage centrally provided resources where they make sense.? Examples include policies; security implementations and services; Governance resources; Publishing; Caching etc.
• Enabling teams to override inherited processes where necessary.? Create exceptions and implement alternate approaches where exemptions have been sought.
• Create hard isolation boundaries. For example payments processing in many organizations represents an extremely sensitive area warranting a much high security posture and governance model.
• Centralized governance providing visibility over the entire estate and applying API standards across the enterprise with the ability to track and monitor divergence from standards as well as exemptions.
• Support special cases of federated API management where a completely different and isolated API management model is required from the rest of the enterprise.? For example where an organization is developing a product E.g. Development of an underwriting workbench.?
• Supporting multi cloud strategies that allow companies to leverage best in breed tooling from the cloud provider landscape.
• Supporting federated cloud provider frameworks such as Microsoft Cloud Adoption Framework.

API Management Landscape

Centralised API Management

This need for an ‘API management’ layer gave birth to an explosion in solutions - Apigee, Layer 7, 3 Scale, IBM API Connect, Apiphany (which later became Azure API Management) and many more.? Most of these products have been developed in a common architecture with:

• Gateway/Data Plane - a gateway message processor
• Extensibility framework - a plugin architecture enabling the gateway functionality to be extended using low-level native languages
• A control plane to centralize control of these gateways.

Typically these solutions are provided software as a service and many of these solutions are tightly coupled and offer limited and inelegant solutions to today's federated environment challenges.? This architecture looks as follows in large complex federated organization:

Larger organizations with environments that resemble the above illustration and a large diverse service portfolio including diverse API styles and products experience the following challenges:

• Bottlenecks associated with centralized governance and resources.? Tools and technologies do not support more than one governance structure and model.
• Network loops and latencies especially surrounding east-west traffic where internal services require connectivity to other services within the organization's boundaries.
• Difficulties with isolation and problematic customizations/policies degrading the service for all users.

Micro gateways were invented employing both federated and non federated configuration depending on the product.? These were merely a band aid for a deeper architecture problem for organizations supporting decentralized technology environments.? As cloud computing and services became more pervasive with better self service models more organizations are implementing decentralized technology approaches.? A new approach to API Management is required.

Federated/Decnetralized API Management

Challenges surrounding implementing these tightly coupled software as a service API management solutions led to a new breed of API management solutions which provide decoupled components:

• Gateway/Data Plane - Deployed - With a Micro Kernel/Plugin Architecture for Extensibility
• Control Plane - Provide Governance and API Management configuration/components that are then distributed to federated instances following a defined deployment model.

Examples include: Kong, Tyk, Gravitee, Sensedia etc. With this model we see a distribution of functionality between software as service provided control plane and either self hosted or software as service gateway/data plane.? Factors governing whether you self host or software as service come down to latency issues and the proportions of east-west and north-south requests required. This architecture looks as follows in large complex federated organization:

Most of these products are now offering service mesh products in order to enable native connectivity and interoperability with Kubernetes clusters.? Many federated technology environments are deploying workloads in one way or another on Kubernetes clusters taking advantage of container portability. ? Most of these products are built on a micro kernel / plugin architecture enabling each data plane to be deployed with a completely independent configuration and operating model.? This is how they solve the single governance deployment model problem.? These products support a fully federated API management model and solve many of the challenges associated with centralised architecture.? Network loops and latency issues can still be an issue if the solution is not carefully architected and deployed.

Challenges that still exist even in this evolved architecture:

• Latency issues - Centralised infrastructure can still cause latency issues in geographically disperse/multi region organisations.
• Infrastructure costs and complexity - Standing up and licensing needed gateway and control plane components can be expensive and complex.? Many organisations are not good at running software and increasingly looking for services.??
• Engineering approach is moving strongly towards self service just in time in cloud infrastructure approach.? Much development is moving toward co-pilot and cloud based activity.? Local development environments are being considered old school/guard.?
• Gatekeepers and governance models are still centralised and lack tooling to support multi model governance processes.? For example a governance model for partner channels, another for consumer channels and one for internally consumed services.? Some organisation also have complete products that again need their own governance model. The components need to be architected and installed to support these differing models and approaches.
• Lack of real fully automated governance processes that ensure consistency and quality API definition and consumption.

Rethinking Federated API Management

The API management landscape has settled and organisations are choosing between either centralised or federated/decentralised API management tooling depending on organisation nature and approach to APIs and Information Technology in General. The solutions look more or less the same with a few new entrants based on the same concepts.? Also in most cases irrespective of model and tool you choose the total cost of ownership is very similar.? You end up making trade offs between complexity, duplication and flexibility.

The challenges of managing API management

In deployments like the above, the centralised control plane becomes a point of management control. Which is a good thing in an environment desperately needing compliance and governance.

Sadly, it also becomes a bottleneck in your workflow. The gateways are protected by guardians of the gateway: not malicious actors; but engineering and ops leaders charged with a noble task - governance and compliance - that unintentionally impacts the autonomy and velocity of your API program.

Shifting Left

The best engineering teams in the world build tools that support rapid execution and iteration. Allowing their engineers to operate with a high degree of autonomy but appropriate controls to stop non-compliant or problem code making it to production.

This is achieved by shortening the feedback loop. Learn from your mistakes and get feedback faster. Classic agile manifesto thinking. This has been the core facet of process improvement from the Toyota Production System, Agile development to the principles bottoms up processes that support the likes of Facebook.

If you need proof that Facebook was one of the best engineering organizations in the world, just look at their open-source side-gigs that power so much of the internet today: React, React Native, PyTorch, GraphQL, Jest, Docusaurus, Flow, and more…

The other phrase commonly used today for ‘shortening the feedback loop today’ is shift-left. The "shift-left" philosophy, advocates for integrating and addressing operational considerations earlier in the development cycle.

Today’s API management products are shifted right. With the gateway sitting on an iron throne, protected by a team of loyal nights - actively standing in the way of this modern approach to iterative execution and productivity.

The best engineering organisations have a clear approach to getting shift-left working:

• Empower developers to learn, play, build, and test rapidly: provide easy access to all the core components of your infrastructure
• Deploy devops that is used throughout the SDLC so deploying is FAST and part of the everyday flow for your team - something that is practiced regularly not tried for the first time deploying to QA or Staging
• Deliver tools and experiences that enable developers and operations
• Focus on automation, and reduce operational overhead of infrastructure

Shift-left infrastructure has been exemplified by platforms like Vercel and Netflix, represents a paradigm shift in web development and deployment, aligning closely with modern DevOps practices. In this case, it takes IaC (Infrastructure as Code) one step further to something Vercel call “Configuration as Code”. That is: the text files in the repo that define the application are enough to deploy it to a ready state. Beautiful.

API Management - New Entrant

A new entrant into the API management space Zuplo is changing the landscape and approach to API management.? The following diagram illustrates how Zuplo has changed API management using an Integration Reference Architecture evolved from the Forrester Four Tier Model:

Zuplo is implementing much of the API Management, Mediation and Governance functionality as an edge service.? Whilst this has a significant impact on development and how you work with API management functionality it also introduces some new challenges that need addressing.? It also potentially greatly simplifies networking and simplifies this infrastructure.? Let's take a look at a model architecture to see how this introduces some challenges:

So we gain as many environments as we want and the ability to have a fully decentralised control plane as an edge service.? This works for developers and internet inbound and outbound north south traffic. But what about API management and traffic for service to service east west traffic/communication where the services might be in cloud provider subscriptions or VNet peered cloud provided networks or networks available via a Transit gateway. ? For these use cases we have to go all the way to the edge and back in again including the full set of associated latency.

Edge Based API Management

With Edge based API Management Configuration as Code the following benefits exist:

Benefits

• Support for an infinite number of cloud hosted API environments is a huge benefit for development.? Engineering and local testing can be done with the full blown toolkit and management platform.? Effectively completely decentralising/federating the control plane and the needed processes surround it.??
• Deployment is as simple as committing and merging with your favoured source control management toolkit.? Github, BitBucket, Gitlab etc.
• Brings the deployment of API orchestration closer to the service source code.
• Moves the deployed system out of the connectivity zone where changes in configuration and setup can be hampered by the network gods.
• Easier to standardise development environments for all developers and to enforce development standards across entire development efforts.

This approach can be thought of as everywhere, all at once, all of the time and is greatly improving developer productivity and lowering the developer skill level bar.? Development is getting ever more pervasive and seeping into a spectrum of business roles such as reporting, actuary, data scientist to name a few. Development is becoming more accessible with less requirement for specialised/expensive skills and tools.?

Disadvantages

• Does not provide a solution to the service mesh problem.? Solving the internal service publishing, discovery, security and governance problem.? Does not provide an API management solution for internal services/APIs at the process and system layers as illustrated below taken from: https://blogs.mulesoft.com/api-integration/patterns/patterns-to-debunk-api-led-connectivity-myths/. Zuplo as an Edge Base solution solves the Inbound Outbound North South traffic however there are limitations when it comes to East West traffic for which other solutions will need to be used to cover these gaps.

Depending on your organisation structure, nature and size this problem may not be significant in your environment.? However if this is a significant part of your business Zuplo may not be a good fit as you will undoubtedly have to supplement it with other toolkits in order to make the overall approach work.? Conversely if you have solid engineering effort you may have already invested in supplementary toolkits and Zuplo may be a good option for managing your external interfaces.

This edge based approach aligns with the evolution of the engineering process away from localized development environments towards cloud based development environments. ? We already see this becoming pervasive in the engineering process due to:

• Standardized cloud hosted virtual desktop environments;
• Increasing number of software as a service engineering toolkits such as Databases, Continuous integration and Deployment Pipelines, Artificial Intelligence Processes, Data Pipelines, Data Notebooks just to name a few.? All of these processes have limited ability to run isolated and localized versions for a range of reasons.
• Increased cloud based artificial intelligence development support tools such as co-pilot means much more development is being in browser based thin clients and being deployed directly to low cost transient full blown cloud platforms with pay by the minute consumption models.
• Maturity of static analysis tools driving fewer iterations before having correct working code driving down the need for isolated and localized environments for experimentation and testing.?
• Maturity of the self service infrastructure tooling enabling developers to establish their own cloud based environments.

Summary

Careful consideration of your environment and maturity levels with a good eye on the future of technology needs to be done before selecting tooling to solve problems.? Zuplo as a new Edge Base solution provides a good solution to the North South traffic problem, however with limitations when it comes to East West traffic for which other solutions will need to be used to cover these gaps. API management is now a competitive environment with established costs that are pretty well understood.? Also in most cases irrespective of model and tool you choose the total cost of ownership is very similar.? You end up making trade offs between complexity, duplication and flexibility. Careful consideration needs to be given to the level of federation needed/desired and the technology challenges you want to undertake to solve.? Considering the priorities of the use case personas. ? ? ?