Microservices in Digital Business Solutions: Today and Tomorrow

Microservices have been part of distributed solution architectures for several years. But what is their role in digital business automation solutions? Does their role change or become more important? Do we approach microservice design differently as part of a digital transformation? Can microservices be used to support underlying data science system processing?

In this podcast, I'm joined by author and microservice technology expert Eric Barceló, as we tackle these questions and further speculate how microservice technology might evolve to further empower digital business solutions in the future.

Listen to the podcast here (transcript is below):

Also listen to the podcast at?Spotify,?Google Podcasts?and?Apple Podcasts.

About the Contributors

Eric Barceló is a microservice and containerization technology expert who works as an international IT consultant. He is also an upcoming cloud computing book author. His current role is as an AWS specialist with JP Morgan Chase.

Follow Eric: www.dhirubhai.net/in/ericbarcelo

Thomas Erl is a best-selling author and oversees the Pearson Digital Enterprise Series from Thomas Erl. He is the Founder and Senior Advisor for Transformative Digital Solutions Inc. and the CEO and Director of Learning and Development for Arcitura Education Inc.

Follow Thomas:?www.dhirubhai.net/in/thomaserl

Subscribe to Thomas on YouTube: www.youtube.com/@terl

Books by Thomas: www.informit.com/authors/bio/f8d115ad-20cc-4d42-ad99-7e349d34f90d

Podcast Transcript

Thomas: Hi, this is Thomas Erl, and welcome to another episode of the Real Digital Transformation Podcast. Today I have with me Eric Barceló. He is an international IT consultant and an upcoming cloud computing book author. His current role is as an AWS specialist with JP Morgan Chase, and Eric has worked extensively with a variety of technologies and with microservices in particular.

And today's topic is about how microservice technology, how microservice design relates to digital transformation, relates to our creation of digital automation solutions and how should we approach it differently in support of some of the strategic goals associated with digital transformation. But let's first welcome Eric to our show. Eric, how are you today?

Eric: Fine. Thank you very much, Thomas. Thank you for inviting me to your show, and ask away.

Thomas: Yeah, thank you Eric. I'm eager to get into this. I've had my own experiences with microservices. I've written about them in previous publications. And I guess the microservices themselves have become increasingly important and talked about over the years. They themselves have evolved over time in terms of their relevance, not just to digital transformation, but just to the average automation solution that organizations are building.

You have extensive history with them going back several years. So what's important with microservice modeling, microservice design is ensuring that the functional scope is truly targeted to its role and doesn't bloat into a scope that would no longer qualify it as a microservice, and that allows us to leverage its smaller footprint to support the scalability and reliability aspects of it that we look to gain from it. Is that a correct statement?

Eric: Yes, absolutely. That is a perfect definition of what a microservice really is.

Thomas: And it's then important for us to make a distinction from an architectural perspective that these are microservices, these are modeled, designed for that particular scope, and that also affects how they're implemented and what technology we might use to isolate them so that they can carry out their high performance, high reliability responsibilities and distinguish that from other services that don't fall into that category because those will be designed differently and may also be implemented differently.

Eric: That is correct and, and most of the services that you will need to implement a bigger solution are usually not microservices.

Thomas: Let's talk about microservices in digital business solutions as part of digital transformations, which can be large in scope, which can involve new and foreign technology that we may not have had as part of our IT enterprise previously. How does that new environment affect how we design microservices? How we position them, how we utilize them? Or is there an impact?

Eric: Oh, definitely. Let me tell you, Thomas. Nowadays when we talk about digital transformation, we usually want to ensure that our organization is adopting this new paradigm in order to be able to provide a better service to our customers.

Customer centricity is one of the fundamental reasons why organizations are going through a digital transformation process. In this direction, microservices allow us to present solutions and functions and capabilities to our end users that are really performant, reliable, always available, that can scale up to the level of volume that we might have, especially nowadays that most of the services that organizations offer are provided via the internet, which is an open door for enormous volumes of customers coming in and at least inquiring about our services. Of course, if they actually buy our services and become our customers even better, but then that represents a volume of service that we need to be able to provide with the required levels of reliability and performance that will keep those customers satisfied and coming back all the time.

And that's where microservices actually fit some capabilities might normally or, or usually, or currently be very slow in our systems, and if that is the case, then we need to apply the principles of microservice design to ensure that those capabilities can reach the levels of performance. Of course, those principles require microservices to be designed in ways in which they will need dedicated infrastructure, dedicated resources, dedicated bandwidth, for example.

All of those requirements make a microservice bring with it complexity in our infrastructure, in our architecture in general, and that is why you have to be careful in incorporating only the really necessary number of microservices that will allow you to provide an overall satisfactory level of service to your customers, to your consumers, without exaggerating in the amount of additional resources that you would have to implement.

Thomas: Sorry, Eric. So yes, you, you are associating the utilization of microservices strategically as part of our solution architecture in support of our goals with digital transformation to enhance customer centricity, to enhance the customer experience by having microservices positioned in such a manner that our online presence, our online solution, performs consistently regardless of the volume of customers that may be concurrently accessing it, and by virtue of having that dedicated infrastructure, we can enhance customer centric aspects of our solution, by just making sure things run always as well and as performant as possible. Let me know if that's correct and if that is correct then, is that it? Is that the role of microservices or is there more that we can use them for in support of our digital transformation goals beyond improving reliability and scalability.

Eric: Right, that is it. You paraphrased the concept very clearly. That's what microservices’ role is in digital transformation process mainly.

Of course, you have to identify specifically where there are functions and capabilities that can benefit from a microservice architecture to provide these levels of performance. Sometimes they are not directly going to be providing a capability that touches the customer in the end, they might be used as well in, for example, big data architecture and big data analysis processes in which we need to enhance the performance of these big data analysis procedures because we want to provide a result that, again, focused on the customer, on the end user is a quick result, a result that will really be there, maybe even sometimes in real time. So in real time analysis, microservices may also provide an architecture that allows for the capabilities required in those kinds of procedures.

Thomas: Since you brought up big data systems, the use of data science technology in general as part of digital business solutions, working with large volumes of data where the processing occurs, whether it's within the organization, within a cloud environment, using that data, that data intelligence for human decision making, for automated decision making. So what you just described is about responsiveness in relation to the delivery of analytics results of data processed through data science systems to the front end. Be it customers whose experience may be impacted by the stream of real time information that we're flowing back to the solution or to a front end that might be for internal access by others that also need access to this information.

So you can see an opportunity to utilize microservices in those particular functions of the solution as well.

Eric: Yes, definitely Thomas, that there's an opportunity there that will allow the organization to provide these results to our end users with the level of reliability and performance that will keep them satisfied and coming back to us for our services.

Thomas: So the criteria is whether or not that delivery of data intelligence requires high performance and reliability. Cause sometimes it may not, if a manager has to sit and wait for 30 or 60 seconds to get some results as opposed to, an end user for whom the results may be critical to some task they're performing and they need the results in a matter of milliseconds. Those are the types of scenarios we look at to determine whether or not. Incorporating microservices within those automated tasks is justified, and then along with the complexity and the infrastructure investment we need to make in order to put that in place. I know that might be oversimplifying it, but is that sort of the gist of it?

Eric: That is correct. That's precisely where we have to consider the benefits of incorporating microservices in our architecture. Balanced with the complexity and the cost that it might bring. So when it's justified, we should do it. If not, then let's continue to use the technologies that are in place. By the way, there's something also very important, some infrastructural developments like continuation for example, sometimes it's confused with microservices in the sense that some people recognize containerization as an exclusive environment

For running microservices or confuse programs running in containers as microservices and both concepts are wrong. I mean, yes, containers are fantastic environments for running microservices because they will allow microservice to reach it's levels of performance and scalability even better. But containers are not limited to microservices, so we might sometimes be able to simply use containers to deploy services that are not necessarily microservices and allow those services to reach acceptable levels of performance. When that is not enough, that's when we have to identify these specific capabilities that need to be designed and deployed as microservices, because microservices are very special, and we are gonna have to provide those additional resources for those microservices to actually be able to reach the levels of operational requirements that we need to provide.

Thomas: Yeah, containerization is definitely an important topic. I'm hoping to have a separate podcast dedicated to containerization technology in relation to digital business solutions. But right now, as you just pointed out, the perceived relationship between microservices and containerizations is a very close one. It's they, they go hand in hand. Is it now such that microservices are deployed in containers by default? Regardless, you know, for building any type of solution today, would that just be the default approach? Or do you see situations where genuine microservices are required, but the use of containers is not a necessity or is perhaps a detriment to the goal of that particular microservice implementation?

Eric: Yes, I do see some situations of that sort. Sometimes microservices need to provide a service that is so strongly related to the underlying infrastructure that running those microservices directly on a physical host may be better for the performance of those microservices when they don't really need to be highly scalable, in other words.

But containers and containerization definitely provides the best environment for when microservices need to provide high levels of scalability. So you have to identify which specific operational requirement your microservice needs to focus on, and if scalability is in one of them or is part of that, containers are an ideal environment for deployment of microservices. But if not, sometimes physical hosts can directly provide better environments for the performance of these programs as well.

Thomas: Very interesting. I'd like to get back to strategically utilizing microservices in support of our customer centric enhancements, which are often primary motivations for building digital business solutions right now because they do give us the ability to truly distinguish ourselves from competitors. So in relation to that, can you think of a project you've worked on where you've seen that in action? You've seen actually yourself how microservices have specifically benefited the customer experience.

Eric: Oh, absolutely. I had an experience with a company that developed software for the oil industry and, this company, was already offering a product that allowed the oil company to calculate the oil production in their oil fields, which had many oil wells and the volume of calculations required to simulate a production based on possible interruptions of certain the production in certain oil wells or even in full oil fields for repairs or for other reasons. It required a lot of calculations and a lot of accessing databases for data that had to be used in those calculations, and it was taking more than 20 minutes for a single calculation of a production level for oil in a certain region if you will. So there was a lot of data management, data handling, and calculations that needed to be performed.

It was taking too long, and when that portion of the software was redesigned using microservices, then everything changed. The oil company was about to find another vendor and move to another application because this application, although it was providing accurate results, it was taking too long for that, and it wasn't satisfactory for the end user. So by redesigning the application to incorporate a few microservices in this part of the calculation, 20 minutes went down to just a little under two minutes and that became, I know it's still a long time, but it definitely became much more satisfactory for the end users.

Thomas: Thank you for that. So before we conclude, I'd like to go back to a statement where you made that you made earlier about how we need to be careful when we choose what will be built as a genuine microservice, as opposed to as a standard service. And then we build our solution architecture accordingly.

Based on your experience, and I know everyone's experience can vary, but based on what you've seen, what is a typical percentage of a solutions technology services being or requiring microservices? So if we have a hundred services we need for a large scale digital business solution that establishes our online presence, runs our business automation. Of those hundred services, typically, what would you expect the percentage to be microservices?

Eric: Yes, Thomas, I would typically expect from five to 10 microservices at most. Yes. I mean, some companies out there are dedicated to providing services with very high levels of performance and reliability that require very high processing capacities. And I'm talking, for example, streaming service companies and things like that. Those well, obviously require many, many more microservices than what I just mentioned. But there are not very many companies like that. Most organizations will have between five and 10% of their services that are part of their service oriented architecture, uh, designed as microservices. That's the typical volume.

Thomas: And when we're saying that, we acknowledge that in some organizations, the term microservice is used loosely and is not always used as a label for services that truly have a narrow and targeted functional scope?

Eric: That's correct.

Thomas: So in some organizations they may say we have over 50%, or they're all microservices. But regardless of what you refer to your services as, what you are pointing out is that those services that do have a deliberately narrow. Modeled functional scope that we give high performance, high reliability responsibilities to, and that we most likely need to isolate into dedicated autonomous implementation environments, and we invest in the infrastructure accordingly to provide that. Those are the services that you've seen require about five to 10% of our overall service allocation.

Eric: That's correct. Exactly.

Thomas: I wanna point that out because however you label things, that's a matter of semantics and if you want to call everything, you're building a microservice. That's your prerogative to do so. But what's important is that from a technology architecture perspective, we make that distinction between those services that have those high end responsibilities that justify the investment, that justify the complexity and those services that fall into other categories. And I think that's a really important distinction to make as you've already pointed out.

So Eric, I guess one last question that comes to mind is the future of microservices. Are we satisfied with how microservice technology has evolved? Is it doing the job we needed to? Now that we are deep into the digital era where we have these new functional requirements, where we have perhaps more high performance, high volume requirements than we've had in the past. Where we're incorporating data science technology to enhance our presence to enhance customer experience. Is microservices matured? Have they fulfilled the role we've expected, or do you see more on the horizon?

Eric: I mean, agility is one of the important requirements from organizations. It means many things, but it also means continuous capability for evolving and evolution requires every time more technology.

So I do think that the requirements for which microservices are usually incorporated into a service or internet architecture are going to continue to grow and evolve, and that both the number of microservices, the amount or percentage of microservices, if you will, is going to increase, and the technologies that will be the underlying technologies for the development and deployment of those microservices are also going to evolve and increase. I would like to think that in the future technologies that are in development, like maybe even quantum computing will be used extensively as part of the infrastructure for underlying microservices, particularly in those organizations that really require very, very high levels of performance in very specific capabilities, if you will.

Thomas: What about from an economic perspective? Can we reach a point where microservices and the implementation environments they require can become more commoditized? So, you know, complexity is one aspect, but often it is the investment that causes some organizations to limit the extent to which they utilize microservice technology in isolated autonomous environments, dedicated servers, dedicated infrastructure, allowing them to do their job.

But can we reach a point where, you know, even if I don't need it now in my other 90% of my solution architecture, why can't we get to a point where I build everything as being high performant so that the solution architecture is as accommodating and flexible for future usage requirements for future unpredictable scenarios that it may need to deal with as opposed to this is the solution architecture that's good for today. A year from now it's being used differently, or there's new usage volume it was not designed for, and I have to pull another 10 services away from my other category and implement them as microservices to adapt to these new requirements. Could we reach a point where we just build our solution architecture from scratch with a hundred percent or a very high percentage of high performant autonomous, independently implemented microservices to be ready for anything? Is that something realistic at all?

Eric: Yeah, I mean, I would say it is realistic. It would take a lot of work though because you know, decomposing services to turn them into groups of microservices is another part of the complexity of bringing microservices to the table. But if you have the time and the resources to do it, I guess you would be preparing your architecture for the future. Definitely…

Thomas: Right. I'll just interrupt you, Eric. So what you just stated, that's, that's a complexity resulting from migrating from our existing standard services toward a higher percentage of microservices, and that I understand that, but is it at all feasible to think about building a new solution from scratch? With 80 or 90% microservices right out the gate so that we build something that's ready for anything. It's not something we want to do now. Because of everything you just stated, that would prevent and constrain us from doing that. But I'm just curious if you know, three, five years from now the technology surrounding microservices in cloud environments in particular, could it be commoditized and become commonly accessible and highly affordable, I should say?

That's just the way we build our solutions. They're high performant from the beginning because it's cheap and easy to do so. And we build solutions that are much more flexible and adaptable to unforeseen usage change. Is that vision of the future something that you see as possible, or your understanding of the technology required to achieve that do you think that's improbable?

Eric: No, no, I, do think it is possible actually. We, we see that happening with the evolution and the level of adoption of newer technologies like containerization, for example. I think it is possible to do that as you describe it. However, it has to be done with care.

That's my only concern because, again, the complexities that are not only infrastructure related, but also governance related from the standpoint of keeping your architecture under control from a standardization standpoint is much more difficult if it is divided into many more parts than originally or before, if you will.

So you are gonna have many more moving parts. There's, there's also an engineering concept about how a system will fail more frequently depending on the number of parts that the system is comprised of. So again, having more services as microservices from the beginning increases the complexity and the difficulty of bringing your application to a production environment. So you have to be careful.

Thomas: Sorry to interrupt you again, but I, I like what you just said. Having more services as microservices because having more services as microservices results in a higher quantity of services in the end, because as you pointed out earlier, those services need to be decomposed. You might have regular services now, and if you then decided to migrate given service into a microservice architecture, that may end up in three separate service implementations. So all of a sudden, one service has become three. And if you do that repeatedly, you increase the quantity. And by virtue of that, the complexity of your overall solution architecture.

Eric: Right, and the possibility that failure will occur more frequently, for example.

Thomas: And that leads me back to the technology, you know, I'm just curious if, if that's where we're heading ultimately, you know, we began with monolithic applications client server applications were considered complex when they were first introduced compared to monolithic applications.

Eric: Right.

Thomas: And then distributed three tier and tier solutions introduce more complexity, but also more benefits. But at the time, were considered somewhat revolutionary, but also more complex. And as we now progress into highly sophisticated digital business solutions, I'm curious if ultimately we’re moving toward an environment where it'll just naturally be X amount of tiers of high performant services, whether they're called microservices or not at that stage, who knows? But that'll just sort of be the default environment that we build regardless if we need that power, that processing power to begin with, it's something that we just naturally build.

So we have extremely flexible, highly adaptive, highly agile systems that can automate our business in, in all kinds of ways without us having to reengineer them, redesign them, and whether the economics behind the infrastructure will allow us to do that by default and whether there will be additional tools or fail safes or, or other types of built in mechanisms to also accommodate not just introducing more single points of failure, but also to accommodate the complexity of it. To perhaps abstract some of that complexity for architects designing it so that they don't have to worry about a lot of the interaction and low end, low level tech, you know, details as to how the services communicate, but rather they get an abstracted view that they can work with from a business automation perspective and all of the other complexities are, are dealt with by the underlying environment.

I'm just curious if that's where we're heading and I don't know if it's containerization alone that can accomplish that, or whether it's a second or third generation of microservice technology, maybe additional infrastructure technology. But that reality to me seems like something that would benefit where we're going with our digital business solution requirements to establish those types of more powerful and sophisticated environments without us having to make hard choices about is it justified? Is it too complex, is it too expensive? So that we reach a point where it's so commoditized that we just do it by default.

Eric: Yes, I totally agree with you, Thomas. I think that is a very visionary perspective of the future, as you said, whatever we call microservices in the future or these types of architectures as they evolve. I think that would be very desirable. And there are many technologies that would have to converge in this, in this process, including cloud computing, making things more accessible and, and economically feasible is something that can be brought through cloud computing very, very strongly. So these huge cloud providers, are going to be able to, and they're already starting to do it, for example, incorporate expensive technologies like quantum computing, which I mentioned earlier.

And quantum computing could be a source of high performance for any level of fragmentation that you might want, you know, envision in the future for applications so that every single micro feature of your solution has the potential level of performance and scalability that you might require, and you don't really don't have to worry about the underlying infrastructure. I agree that, that, that is something that we should be able to see in the future as the different technologies are evolving.

Thomas:?So far, you know, that sounds really interesting. Quantum computing would be a great topic to discuss and have a dedicated episode for. Eric. I'd like to thank you for your time today, this has been really enlightening and insightful. We'll look forward to speaking with you again in the future. Thanks so much.

Eric: Thank you, Thomas. ?

Please note that although this transcript was professionally prepared, it has not yet been reviewed by the contributors for accuracy. Please report any errors to the newsletter editor.