Service Mesh Explained in 5 minutes for Developers

Service Mesh compared to AOP and Servlet Filters - Cross-cutting NFR implementation with Istio

Service Mesh implements non-functional requirements (NFRs) that add to the user experience through increased quality and reliability of the service. This quality of service is essential for your company's brand image. Service Mesh adds operational support, increased security, and operational predictability through implementing NFRs as cross-cutting concerns that are similar to the techniques in AOP and Servlet Filters but at the network data layer. (AOP is defined below, but it is a way to add behavior without changing code. NFRs are also defined in more detail below).

Istio, a popular Service Mesh implementation, decorates the network layer to implement cross-cutting concerns which are usually NFRs. 

• Istio Service Mesh is to MicroServices as AOP is to DDD (domain-driven design) and OOP (object-oriented programming).
• Istio Service Mesh is to MicroServices as Servlet Filters are to Servlets. 

AOP (Aspect-oriented programming) is a dynamic decorator design pattern. The decorator design pattern adds additional behavior to an existing class by wrapping the original class and implementing the same interface, then delegating to the original while adding additional behavior before and after the delegation (or even skipping the delegation if apropos). Servlet Filters (from Java Servlets a framework for developing server-side Java web application) is a programmatic way to intercept call handlers to HTTP methods like GET, POST, etc., to add additional behavior using the same decorator pattern. Both Servlet Filters and AOP allow the transformation of parameters to the call sites (headers, body, etc. for filters and method arguments for AOP) and allow the decoration of the response (the response for Servlets and return value of the method for AOP).

NFRs are important to your companies image and brand. NFRs define application requirements like security, reliability, performance, maintainability, scalability, and usability. NFRs define system qualities that are as important as functional capabilities and features. See Scaled Agile's description of NFRs for more detail. For NFRs importance, imagine a customer's order getting lost due to an outage. Or, a website for a popular sport going down on the biggest game of the year. NFRs are usually in the realm of SRE (site reliability engineers), DevOps and Infra. NFRs can be InfoSec related, for example, avoiding data breaches that release customer's personal information. Think of NFRs as things that are not in the business domain per se but will damage the brand of your company. Product managers are great at specifying business requirements but usually, NFRs are more in the domain of technical product managers or software architects.

Aspect-oriented software development defines crosscutting concerns as aspects of a program that affect other concerns which are often NFRs. These concerns are added behavior that can't be cleanly decomposed into objects and methods like security checks, logging, auditing, tracing, transaction propagation, etc. Thus instead of spreading this code throughout your codebase (code duplication), mixed with business logic, and tying business logic to other systems, then you use AOP or servlet filters to decorate these call sites (call site is a class method call or HTTP method requests).

In a similar way to AOP, Service Mesh handles concerns like circuit breakers, retry logic, tracing, policy enforcement, and mTLS (mutual TLS which is used in zero-trust networking). Before Acegi, which became Spring Security, it was very common to implement SSO and security checks in Servlet Filters. Acegi implemented Security/SSO integration via Servlet Filters and Spring AOP.

Service Mesh allows developers to focus on business logic while the crosscutting network data layer code is handled by the Service Mesh. This is a boon because this code can be tricky to implement and hard to test all of the edge cases. Service Mesh takes this a few steps further than AOP or Servlet Filters or custom language-specific frameworks because it works regardless of the underlying programming language being used which is great for polyglot development shops. Thus standardizing how these layers work, while allowing teams to pick the best tools or languages for the job at hand.

Just like AOP and Servlet Filters are often used for cross-cutting NFRs so are Istio services via Mixer. The Mixer is Istio's way to mix in additional behavior through custom extensions which can decorate the network data layer.

The cross-cutting services that Istio provides are mTLS, rate limiting, support for service identity, circuit breakers, identity via SNI (TLS Server Name Indication, attributes from SSL/TLS), identity via JWT (JSON Web Token used for claim/identity token), RBAC (RBAC is role-based access control) and more. Istio provides a growing set of cross-cutting concerns. Could you add more? Yes. Yes! You can. You can write custom Mixer adapters.

For example, a cache service at the Istio Service Mesh level would be a cache like an HTTP cache (or a cache on top of another level 7 network stack). In fact, this is not far fetched as they are working on it now: Istio/Envoy HTTP Cache WIP. Again:

• Istio decorates a network stream as AOP decorates a method call. 
• Istio decorates a network stream as a Servlet Filter decorates an HTTP request/response.

There are HTTP caches like Varnish and Squid but there are plenty of caches that were implemented via AOP and Servlet Filters interacting with the cache of your choice (local, JSR-109, RocksDB, Redis, Memcache, etc.) and now you can see the equivalent emerging in the Service Mesh space.

This Istio decoration includes transforming the input and output just like AOP and filters. Istio has ways via selectors to pick which services to decorate the same way AOP can decorate methods or a Servlet Filter can select certain Servlets. 

Istio works with Kubernetes and there is work for integration with VMs as well as Consul/Nomad (Consul is a service discovery software which is often used with microservices, and Nomad is a container orchestration software similar to Kubernetes or Mesos/Marathon). You can extend the Istio Service Mesh to other Kubernetes clusters using a shared control plane or independent control planes. You can also add networks and services via service entries, network CIDRs (a way to define networks or subnets for Internet Protocol addressing), or using its service registry interface which has implementations for Consul and Kubernetes.

If you understand AOP or Servlet Filters, then you also understand Service Mesh support for cross-cutting concerns which are also NFRs (see more details on Service Mesh). Istio Service Mesh is a great way to implement cross-cutting concerns for NFRs.

Feedback:

Fadi Maalouli, Senior Software Engineer at Capital One

Looks good! A quick feedback would be that you are assuming the reader knows some important keywords that are used in the article. Like mixer, for example. What is a mixer? I will give a better feedback when I get to read it with more attention :)

Thanks, Fadi. I added a whole section to explain it better. Let me know if that is better. I also added more explanations for many topics --Rick

_____

Nick Lesiecki 

Senior Software Engineer at Snap Inc. (and author of Mastering AspectJ)

Thanks, Rick! I'd never seen the Service Mesh concept explained so clearly before! And as the author of Mastering AspectJ, I found the AOP analogy very apt!

Thanks, Nick! High praise coming from you! You forgot more about AOP than most people know. Thank you, sir! --Rick

______________

Matt McLean

Director, Distinguished Engineer, Data Management Architecture at Capital One

I see the similarity. Personally, I've seen AOP be abused. Scattering business logic across too many places unnecessarily. If there is a cross-cutting service, then I rather see it extracted from the microservice entirely and implemented as a sidecar.

Good to hear from you, Matt. I miss working with you. I value your thoughts and opinions. AOP has been abused. Everything has been abused DDD, XP, AOP, OOP, etc. It is hard to protect the world from the bad use of good ideas. Service Meshes are implemented as a series of sidecars (Istio and Consul use Envoy while Linkerd wrote their own proxy sidecar using Go and Rust). I think they just facilitate using a sidecar proxy so you can extend and decorating easier w/o configuring your own Level 4 / Level 7 proxy. --Rick

__________________

John Katsnelson

Senior Director, Platform Product Engineering at GlaxoSmithKline Pharma GmbH

I like the comparison to the decorator pattern :)

John gave me a lot more advice offline that I used. Thanks, John. Thanks for your input on NFRs too. --Rick

______________

Dean Chen

Sr. Director, Data Platform at Skillz Inc.| eSports | Hiring Data Scientist, Data Engineer, Backend Engineer, DevOps/SRE

Good comparison, makes it very relatable.

Thanks Dean! --Rick

______________

Angad Salaria

Director of Engineering at QOMPLX, Inc.

This is the first time I’ve heard this analogy and will say it’s a (really) great one - (specifically) for the OOP & JVM audience. Love it!

Just today at work, the team solved for a resiliency problem in the stack of a system by implementing a retry pattern at the mesh layer. Totally apt analogy - speaking of AOP and cross-cutting concerns.

Thanks, Angad! Your team seems like they are way ahead of the curve. Knowing you that does not surprise me. I would like to hear more about what your team is doing. Let's meet-up next time I am in your town which will be this month.

--Rick

______________

Scott Johnson 

Senior Manager, Software Engineering at Capital One | Hiring Data Engineers in San Francisco

Coming from a Java background, this was helpful for me to understand the concept. Good analogy and good post.

Leo Przybylski

Lead Software Engineer at InVision

I like the AOP analogy is very poignant. Breaking the features of Istio down as aspects that can be applied as crosscutting NFRs makes the proposal less intimidating. As someone that has had to implement circuit breakers, tracing, and gateways, I can say I would much rather build with a service mesh in mind (AOP) rather than having to unravel a tightly coupled version of it built internally.

Thank you, Leo. Brilliant. I tried to include those points about internal NFRs and the cost into the article. I did not call it out before. See if you notice the difference. :) Good points.

--Rick

Rick Hightower, this does seem like a precursor to a more in-depth series. --Leo

Let's collaborate on a series then. :) --Rick

I thought more would have been on board with the analogy. Obviously, it was intended for a specific audience. Thinking back, the existence of the article is indicative of the way we build apps has changed. There is less emphasis on managing infrastructure and more on making it powerful and flexible enough to meet our needs. We want architecture to fit patterns like we build software. It makes sense.

Our goal should not be to sweat details about the architecture, but to apply a pattern like we would in software. That is the essence of a service mesh. --Leo

Awesome. I love it. Great feedback. Interesting thought too. I tried to address some of that in my updates, but I need to work hard to capture your powerful thought here Leo. Damn. I like it. Well, put Leo. --Rick

_______________________________________________

Shailesh Patel

Sr. Software Solutions Engineer / Consultant

Interesting. I see your point with the analogy with AOP cross-cutting NFRs, but for me, I think there's a big difference with an implementation that clouds this perspective. AOP provides a way to implement cross-cutting code, but service mesh works on actual runtime process management.... See what I mean with the implementation difference? Great summary through the analogy, for those of us that came from AOP and filters way back, when ??

Dude! Thanks for your feedback. I sort of miss your point about the difference but I will see you in two weeks so let’s grab lunch and a notepad or whiteboard and talk. I’d love to get your perspective.

--Rick

(I will highlight and try to address after we talk.)

Gilmar Roberto Maciel Junior

InfoSec

For me the comparison is valid, and I viewed it that way. Even more, I think that the time to devsecops team to evolve in Istio is less than evolve in Spring and Devsecops only need to pass for the DEVOPS team how the security works in ISTIO.

Wow, that is a very good point. Let's chat again in person when you get back. I'd love to capture that thought some more in the article. I took a few stabs at it. See if you can see it. It needs more.

--Rick

________________________

Warner Onstine

Director of Engineering at Estech Systems, Inc. (ESI)

(It) definitely makes sense and makes it pretty clear what Istio is.

Thanks, Warner. I am giving a talk on Service Mesh at CTO Forum next month (March 2020). Let me know if you have any more ideas. I always love your feedback and it has been a while since we geeked out. I did enjoy our discussion last week. Let’s figure out some ways to collaborate. I remember when we wrote some courses together in the past. That was fun. We should find something to collaborate on in the future.

--Rick

______________________

Edward Borukhov

Problem Management | SRE/DevOps | SQA

Nice article, but it can be little confusing for an audience who is not familiar with Service Mesh. Maybe elaborate more on some of the terms like mixer etc.

I look forward to chatting this Sunday too. In-person! I’d love to hear what you have been doing with Kube. You are the man Ed! I agree. I need to make a second pass at this. ... Done. I added a Mixer description. Let me know if that is what you had in mind.

--Rick

________________________

Paul Giles

Technology Leader

"similar to the techniques in AOP and Servlet Filters but at the network data layer" This is a fair comparison. I can also see how reverse proxies (Ambassador) or proxies, in general, offer similar capabilities and thus could be used as a generalization in your comparison. For the mesh, the key distinction is that belongs to the network data layer (plane).

I think I can cover that here or I have a longer more detailed version of this article. Let’s chat. I confess not to know as much as you about Ambassador wrt CC and NFRs but I’d sure like to hear more. I thought Ambassador was more of an API gateway. I need to look again. Thanks again Mr. Giles!

--Rick

_______________________

Bilal Clarance

Senior Director of Data Engineering

Nice little read Rick! Feedback: For folks that don’t know what AOP, which I doubt the majority of DevOps engineers do I would describe a bit what that is and link to the first AOP instance in your article as opposed to your second. ????

Very good point. I will describe it. .. Done.. Take a look. --Rick

____________

Brion Finlay 

Consulting Chief Technical Officer, CTO | Vice President | Director of Software Engineering | Strategic Technology Leader

I suggest putting examples of NFRs upfront. Depending on the scope of your article, maybe take a specific example and describe how it might be implemented with servlet filters, then how it might be implemented in AOP, and then explain how Istio service mesh makes that same example easier by implementing it outside of the app code.

I like that a lot. That would take me a bit to do. But it is a good idea! I took an early stab at it. I think like Leo said that this could be a series of articles but one with some examples would be nice.

--Rick

________________________

Saurabh Raisinghani

Vice President, IT Application Development, Direct Channels at Marriott International

(An) an interesting comparison to AOP or Servlet Filters, although Istio is used in the micro-services world for micro-services NFRs, I am not truly sure if the comparison holds all true. Eg: servlet filters were used for common business logic as well, such as Auth token validation, etc. Istio won’t be used that way. Is it?

It is funny because you put the idea into my head. We were in a meeting and you mentioned cache as a service and how can Istio support that. Which made me think "well only if a cache is a cross-cutting concern (which made this idea CLICK)" which made me think of my work with caching libs, servlet filters, and AOP. That moment when you were asking about Cache and Istio is when a lot of these ideas popped into my mind and I googled this article which cemented the concept further. I think there are a lot of similarities in they decorate and add behavior. The ability to transform the input and output. The ability to redirect. The ability to decorate new behavior. I would think if they can implement an HTTP Cache via Envoy/Istio and Mixer adaptors then yeah you should be able to do an auth token validation, but unlike Servlet Filters, you could apply it to Microservices written in Python, Ruby, Java, JavaScript, and I added more descriptions of examples in both - please check it out and see. It may not be a 100% comparison, but it appears like Mixer Adaptors could do things like SSO, Auth Token Validation, etc. and it is at least a very similar concept. Case in point, right now, Istio allows traffic management and RBAC policies based on SNI, HTTP headers, and cookies, etc. Perhaps as I learn more, I won't think this is the best comparison. At the moment, I do. --Rick

__________________________

About the author Rick Hightower:

I've worked on API Gateways for large music streaming companies to rate limit partners via OAuth vendor ID (as in implemented from scratch and used Consul for clustering and Vert.x/QBit as the implementation), and have worked with Service Mesh based on Consul and ha_proxy (proxy similar to Envoy which Istio uses) before (as early as 2015 before the term was coined). In the past, I've also worked on JSRs (Java specifications) for the Servlet spec., Enterprise Cache (JSR-109) and the distributed data grid JSR. JSR-109 involved intercepting call sites with AOP using Spring AOP and CDI (CDI1, CDI2) (another framework by Java EE that is used for intercepting method calls which also has support for dependency injection and limited AOP) as well. I worked on the RI (reference implementation) for JSR-109, specifically the CDI interceptors (CDI's version of AOP). Lately, I was managing a team using Kubernetes to implement stream processing and transformations engine (Kafka, Scala, microservices), and more recently have been focused on Kubernetes (GKE, Red Hat Open Shift), API gateways, Anthos, Istio, CI/CD, git ops and service mesh as they relate to digital transformation and cloud-native transformations. I did not come to this analogy until working with Istio Service Mesh for quite some time. I will share that story another time. I also have a lot of experience implementing container orchestration and 12-factor deployments which involve implementing a lot of features that Istio provides out of the box. I have also written about MicroServices and have written Microservice and high-speed streaming frameworks for fun and profit and done my fair share of DevOps, Ops, Automation, etc. as well as a circuit breaker frameworks. I love this stuff and very excited about Kubernetes, Service Mesh and Cloud-Native computing.