Reducing your risk by moving to a modern SCM

My title may seem exactly backwards, but based on the number of, and type of organizations that have moved to modern SCMs including full CI/CD pipelines from the old library managers, if you want to be able to keep up, you’re going to need to move. Before you say I have lost my mind, remember for the last 10 years I have been working with companies to move. I am not the only one. There are a number business partners as well around the world who have been steadily moving clients. If I take a look back, the leading insurance companies, many of the largest financial institutions have moved to either RTC or Git. And if I look more closely, many of those that are driving innovations are those that have moved. Organizations see their internal development activities as a competitive advantage in some cases so we can’t share the entire list, but a small subset include, State Farm, Allstate, and USAA, State Street, Rabobank. That’s a small set that I know and that have spoken publicly, but there a ton more, enough to keep plenty of people busy moving. IBM moved most things a long time ago, out of library managers in order to provide the level of support our clients need. Many other software vendors have as well.  If these companies have moved, some almost 10 years ago, what’s the secret? Understanding what the Library manager is actually doing for you, and what capabilities you already have in your existing pipelines is a primary step.  

Start with what a library manager actually does. First it manages your source code, COBOL, PL/I, Assembler, JCL, REXX, unfortunately probably not all your system code as well, but it’s usually the largest percentage of your application source. It provides a level of versioning of the source depending on the solution, and how you have it set up. Second it manages the compile and link process, generating the artifacts and managing the output in PDSEs. It also manages the promotion through the stages and some percentage of the deployment. Usually doing the Binds and possibly the new copies into each stage. The library manager carefully controls the stages so parts flow up the hierarchy as defined. This assumes there is a static set of environments that updates will go through. Organizations have varying numbers of levels, but other than for emergency change, parts go through the stages as required. The library manager usually manages the traditional host languages, but for Java or z/OS connect assets or many other assets those are generally managed by the distributed pipeline and then coordinated with the library manager for deployment maybe.  The Library manager is generally hooked into, or handling the approvals as the changes move from one level to the next.  Since code changes in the same level affect each other, many organizations have multiple of the promotion hierarchies that have to merge together at some point. When items are promoted through the process, the changes have to be retrofitted back into the other environments manually.  

Let’s look at the distributed side, for the purpose of this discussion we will use Git as the SCM though I could also use RTC. I have chosen Git because it’s an SCM and only an SCM, unlike RTC that provides not only SCM function but a bunch of other capabilities as well. Git provides the source code management, usually using one of the many versions of a Git server such as Github or Gitlab. Git tracks all changes for the source code. In the case of Git it is a distributed SCM model, so each person clones the repository locally. They have the full set of code as well as the history of changes, so they can easily see the other changes and changes being worked by others. Git manages version with a hash that makes sure you know exactly what’s part of that version. It’s not just the program, but all the other associated files as well, so each person can be sure they are working independently of any other change. For the build process it generally varies by language. For example for Java you could be using Maven or Gradle or Ant. These build tools do more than just compile and link, they also deal with unit tests and allow the ability to add other additional tasks into the process. The build output is then stored in some artifact repository such as Artifatory or Nexus. Each persons build is done in total isolation from any other build. No data set concatenation to include what ever is there, but a clean build. If artifacts are needed for the build, they are pulled from the artifact repository to insure the correct version is used. There is no promotion process in the sense of a library manager, instead, the artifacts are pulled from the repository and deployed into what ever environment they need to be deployed too using modern deployment tools such as UrbanCodeDeploy. This can be any level of test environment, that could be provisioned just before the application is deployed into it, or an existing environment such as later stage testing or production. The deployment generally does more than just the program output but does the configuration of the middleware as well to make sure the application can run. All changes are included as part of the deployment process. In the case of a container, literally all the parts of the application and required systems are packaged together into an image than is then deployed to run. In the distributed environment this process is usually controlled through a CI/CD coordinator such as Jenkins or Gitlab CI. This process is kicked off based on changes being delivered and each step is run as long as it continues to be successful.  

What are the big differences between the two, first the management of the source. The ability to truly provide full isolation by managing the full set of source. The second a build process that does more than just create the output and that can be easily updated to add additional requirements such as security scanning. The third and in many ways the most major, artifacts are stored and versioned in an artifact repository. This keeps all versions easily managed and easily deployed. This also allows the deployment of an application that is more than just traditional z/OS code, but all of an application at once. This makes it much easier to create new test environments which in the world of shift left testing is important.  This ability to use new environments with an artifact repository is supported through the concepts of infrastructure as code using common tools such as Ansible.  

For the last at least 30 years the exiting library manager has done the job, why is it so less risky to move? What makes these differences so valuable? Simply put the world has changed. I like to use the example of the telephone. 30 years ago we all had house phones, they were attached to the wall via a cord, we did not carry them around with us. We had house phones and work phones, and their were pay phones around in public to use. These phones still work, you could still use only a phone connected to the wall in your house. But do you? Do you carry a cell phone? Cell phones give us the flexibility our world demands today.  The same is true for a Library manager, yes it still works, but it does not give you the flexibility needed in today's market.  

Think about it, do you want to be stuck only with a wall phone, or is it time to look at a cell phone? To learn more about modern tools for mainframe development go to Enterprise DevOps to allow your mainframe environment to move at the speed of todays multicloud world.