Mainframe to Microservices

I'm happy to announce that this month I and four colleagues managed to finish a project that allowed the migration of an application from the mainframe environment to a microservice running in the cloud with a web front-end.

The business rules of the application

It is an application that checks if people with privileged information make the previous declaration of purchase or sale of specific securities in the stock market. The CVM, which is the Brazilian regulatory body for our B3 stock exchange, makes this requirement. That is, as these people could know, in advance, of information that will cause changes in the price of the assets, they need to declare their intention to buy or sell them with a minimum time in advance.

The technology change

The front end with the black and green screens written in Natural (Software AG's mainframe programming language) has changed for a front end made with Angular 9.

The back-end of online transactions ran in COBOL/CICS and batch processing performed in procedures with COBOL, Natural (to access an Adabas database from another system) and Syncsort.

Now, a microservice written in Java/Quarkus, that runs on Kubernetes, expose APIs to our front-end. This same microservice has a schedule that starts its batch process.

As this microservice runs in more than one instance (Pod) on Kubernetes, we created a lock rule that ensures only one instance can run the batch process.

We created a temporary microservice in Java/Quarkus that migrated data from DB2 database to Oracle database. This microservice use an interesting feature of Quarkus that allow the application to connect with multiple data sources using Hibernate. For those interested in this, here's one study project I've created to this purpose:

We use Tibco/Spotfire to make comparative studies of the migrated data and also to evaluate the result of processing the new batch system.

We set up a monitoring microservice that verifies the health of the system, displays the data on dashboards in Grafana and sends automatic alerts if problems occur in the application.

Although the old system was deactivated, there are still some pending issues to be resolved in the new system.

Advantages and disadvantages of the migration

And what were the advantages and disadvantages of this change?

Disadvantages

As a disadvantage, I mainly see the batch process. The mainframe environment provides us with many facilities for this environment, such as schedule, tools for handling files, monitoring and performance that is difficult to match.

Another downside is that the learning curve for someone to develop and maintain a mainframe application is much less than that required for a microservice environment.

The technological changes of the mainframe are also much smaller, a stable application in production can run for years or even decades without the need to be modified.

In the microservices environment, the application needs to adapt to changes frequently.

Advantages

As an advantage we can mention the possibility to choose between some options of programming languages and frameworks to better meet our problem.

The salary of people who work with mainframe is usually more expensive, this is another advantage in this environment.

The application can run on any platform that supports container execution, so we are no longer stuck with mainframes, theoretically now we can run in any cloud provider as AWS, Google or Azure.

The build pipline of the application can contain very complete automated tests that give us confidence that a modification did not introduce a bug.

The deployment process in container orchestration environments, such as Kubernetes, allows us to use techniques to update the software in production at any time of the day, affecting our customers to a minimum.

In the example of this project, this was not so evident, as this was a small system. But for a large system on the mainframe, its division into smaller microservices would bring other advantages, such as the possibility of scaling a larger number of people to work on the application, since independent microservices would not cause deployment bottlenecks.

New team members would find it easier to learn a small microservice than the entire big old application.

Final words

To finish, it was a very interesting experience for me. I was able to use my mainframe knowledge to understand how the old system worked and I had the opportunity to exercise my new knowledge, especially with Java, Quarkus, Microprofile, Docker, Kubernetes, Oracle, Prometheus, Grafana and Tibco/Spotfire.

At the moment, I am studying Angular 9 and I intend to be able to work with this technology soon.

I thank the colleagues who participated in this journey with me, Thiago Vinícius de Lima Pádua, Jo?o Paulo Lira de Sousa, Rondson Da Silva Modesto Lima and Jo?o Pedro Oliveira de Jesus.