My ‘Opinionated’ Cloud Native Journey: Web Servers, Service Proxies, API Gateways, Service Meshes

Here we are for the second part of my 'opinionated' cloud native journey.

If you have not read the first article of the series ("My ‘Opinionated’ Cloud Native Journey: beginning with Messaging & Streaming", you should, in order to get a feeling of the underlying reasons and motivations for me to write and share these articles.

I am juxtaposing quantity with quality. I am advocating simplicity over complexity. I am sharing some insights and my reasoning (sometimes very openly and explicitly, other times a little bit less directly).

We all know what the Web is and what a web server is. If in the beginning Apache HTTP server was dominating (60-70%+) serving the majority of web sites, now the situation is very different. Nginx (acquired in March 2019 by F5, Inc. for $670m) has now surpassed Apache with ~30+% market or traffic share. Another interesting and emerging contender to keep an eye on is OpenResty (based on Nginx and with the ability to run Lua scripts using the LuaJIT engine). OpenResty, Inc. has been founded in 2017 and it employs less than 20 people (12 according to LinkedIn). Why Lua? Because of LuaJIT. Why LuaJIT? Speed (also note: latest release 2017: https://luajit.org/). Business and companies such as Kong or Cloudflare are built on top of these foundations, more on this later.

Web servers used to serve static content, but now you need much more than that: application servers (old school, code has now moved to the front end, on the backend you have microservices exposing REST APIs), service proxies, API gateways, and service meshes. All these components are trying to externalise common functionalities (load balancing, authentication, authorisation, rate limiting, routing REST calls to backend services, canary or blue green deployments, improved observability, security, billing and monetisation, etc.).

Let's focus on the service proxy category, to start with. In addition to the already mentioned: Nginx (+ F5), OpenResty, it's worth having a look at the old (and still good) HAProxy (open source project and respective company: HAProxy Technologies). One more recent and worth having a look at is traefik by Traefik Labs.

However, this category and all projects (open source or not) as well as associated businesses are going to be disrupted by the new kid on the block: Envoy (open source, open governance, Apache License 2.0, written in C++, contributed by Lyft to the CNCF in September 2017 and now a CNCF Graduated project).

I won't even comment on the others projects or companies, but if your business model or pitch to investors is just in this category, it's not going to be an easy nor smooth ride.

Envoy is going to be 'king' here and disrupt everyone else. If you are starting now (either a business or adopting a service proxy, start with running your own tests and benchmarks, invest time to learn and study Envoy and if that satisfies your requirements. Go for it. Safe bet. Conclusion: from 22 down to 1 (the winning or the winner).

Let's move onto the API gateway category. By now, you might have got a sense of where this is going. API gateways and service proxies (or web servers) are intrinsically interdependent and interlinked.

So, in order to pick your own API gateway, I would suggest also to look behind the scenes and check what they are building upon. You'll find that many have chose Nginx (and OpenResty) to build their software stack and business, for example: Kong (6 rounds, total funding $169.1m, ~550 employees) or Apache APISIX. APISIX has been developed by Zhiliu Technology (now API7, total funding ?, ~42 employees) in Shenzhen, China and donated to the Apache Software Foundation in June 2019 and become Top-Level Project in July 2020.?

Others saw Envoy coming and jumped on it, for example: Emissary-Ingress (CNCF Incubating project) by Ambassador Labs (4 rounds, total funding $42.2m, ~90 employees), Gloo by Solo.io (4 rounds, total funding $171.5m, ~180 employees). You won't find these (just yet) in the magic quadrants from analysts... they are not that 'clever' or fast to see disruptive innovation coming).

What about the others? Nah... stay out and far away. They are either too old school, not distributed, not designed for Kubernetes, high resources footprint, too risky, too small, not likely to be around in the long run, open source but closed governance, too high risk of lock-in, no community buy-in, too expensive at a massive scale, etc.

So, here it is: from 20 down to 4 contenders: 2 based on Nginx and OpenResty and 2 based on Envoy. I believe in the long run Envoy will dominate (community over code, open source and open governance, will have the larger ecosystem, free to innovate faster, ...). So if you are starting now and you want to select an API Gateway, check if Emissary-Ingress or Gloo satisfy your requirements and make sure you can do everything 'core' to your business with the features and capabilities available in the open source project. If not, carefully evaluate ongoing running costs.

Here we are down to the last category: service mesh. It's not a coincidence that many of the companies we have already mentioned for API gateways are also providing and working on/with a service mesh: Kuma, for example, is the open source service mesh based on Envoy developed and backed by Kong. Interesting to note that Kong who built their API Gateway on top of Nginx and OpenResty decided to adopt Envoy for their service mesh. Gloo Mesh is the management plane for Envoy and Istio developed and backed by Solo.io. Traefik Mesh is the service mesh backed by Traefik Labs.

However, the two service meshes more prevalent and used (even if their adoption and usage is still quite low) are: Istio (open source and open governance, largest ecosystem and group of backing companies, currently CNCF Incubating project) and Linkerd contributed to the CNCF by Buoyant (total funding $24m, 22 employees) and now a CNCF Graduated project.

There is also a very important shift and revolution happening in relation to networking and related matters such as observability, security, API gateways and service meshes and it is called eBPF. Make sure your selected service proxy, API gateway and service mesh is supporting that or migrating to that, Istio and Envoy do that (for example).

Here we go: from 19 down to 1 (or 5, based on how alternative you want to be). I wouldn't invest (or waste) time or effort with others for the time being.

If you have read until here you might now have come to the conclusion that there are less than an handful of companies and indeed only 2 worth investing on or build your own infrastructure and team using and studying in depth open source and open governance projects such as Envoy, Istio and eBPF, have your team master those and you will go along very well for a long time (perhaps also saving some dollars but taking more time and investment in your own people).