Infrastructure as Code (IaC)

I gave a key note speech on TechM Infrastructure Management Services offsite last week focusing on Disruption in IMS. My main topic was IAC i.e. Infrastructure as code. This topic is getting lots of traction and hence this Blog, which is an extract from my speech.

Remembering the old days

Let me start with some history of IMS since I started in the industry 39 years ago. I am reminded of the saying “What goes around comes around”.  I never saw or even knew where my Mainframe Box was located and worked in USA and India for years working in development & deployment with my terminal. The client desktop was just a dumb green terminal. Deployment, monitoring was all done on the Mainframe  by using JCL scripts written by “infra experts” whom we never saw. I even jokingly call it Cloud without Internet.

When Client terminals became powerful PCs, we created Thick Client applications and used servers only for enterprise systems and data access and created part of the business applications on the desktop. We created complex version management and deployment nightmare when the desktops became large in numbers and disbursed across the globe. Yes, we tried automation but application software patches at a fast pace made version mismatch and created havoc on support with part of the users using older versions.

We also tried to go back to Thin Client model with NetPCs but this never really took off. Now with Mobile and IOT devices we are getting in to similar confusion. With APIs overtaking EAI we are exposing our key business applications to external entities. I don’t even want to think of large number of Micro services accessible to outside enterprise and complexities it will create for the infrastructure management.

Infrastructure & IT Apps were always one in 80’s and 90s. We studied JCL Job Control Language as well as COBOL, CICS and IMS. Most complex programs written for IBM MF were JCL programs managing infrastructure. Same in my Burroughs days with WFL Work Flow Language which has same complex syntax as ALGOL. Best programmers those days were asked to work on JCL/WFL and average guys to work on COBOL!.

Complexities in today’s Infra

Slowly as IMS became a large global outsourcing opportunity, we created this “Chinese wall” between IMS and IT. The small group of people in IMS unit were forced to evolve independently with no support from IT. They wrote Unix shell scripts, Cron Jobs etc. and tried to automate as much as possible and left much of the orchestration to manual intervention.  The huge advances in SW development on the IT side such as Agile, Devops, Design patterns, test driven development etc. was basically ignored by IMS teams.

Companies like Netflix, Facebook, Google and Amazon have pioneered a new generation of Infrastructure and Applications change management. We have heard of multiple production releases per day with no maintenance shutdowns. In addition, we are also seeing many companies especially new age ones are asking their developers to build and run. Developers who create software are releasing and supporting the software in production.  This will involve choosing the Iass on which to deploy and run, load and performance test tooling, monitoring and trouble management etc.

In the good old days, we had 3-4 months of lead time to size, order hardware and 5-6 days of loading the OS and software images. Good SOP and checklists were enough to configure the server in these time scales.  Now with private and public cloud infra, servers are configured in seconds and forcing us to automate e2e.

Start of Infrastructure as Code concept

Origins of IAC concept came when experts in the field such as Kief Morris , Author Cloud Specialist started asking questions.

“Can we treat Systems and devices which host and run software, as software themselves?”

“Why not use tested SW development practices such as Config management, Continuous integration, test driven development, agile, Devops in managing IT Infrastructure?

The big Push came in early 2000 when new challenges were brought to the forefront that shook the technology industry; the launch of various public cloud offerings and the introduction of many 2nd generation web tools.  It became imperative that the current level of Infra management automation and level of sophistication was just not enough to keep up. It is like managing a new Ferrari car with old Ambassador Processes & Tools. Hence, new tools emerged in the market to catch up with the Ferrari and thus came the IAC concept. 

IAC means writing code in a high level language to manage configurations and automate provisioning of infrastructure and deployment of applications and monitoring and the whole nine yards of what we know as IMS. Use proven software development practices that are already being used in application development such as version control, testing, small deployments, use of design patterns etc. Infrastructure as Code is an approach to managing infrastructure that leverages software engineering practices.  This allows IMS teams to make changes far more frequently than they could with old ways of working with increased reliability, security and quality. The essence of IAC is to treat the configuration of systems the same way that software source code is treated.

Tools for Infrastructure as Code

New age and nimble enterprises and almost all startups have embraced open source tools such as Puppet and Chef to implement IAC and getting good benefits. These enterprises already have embraced Agile and DevOps SW practices and hence use of "Infrastructure as Code" was a natural extension. They had quicker implementation and benefit realization. Traditional enterprises have adopted private clouds rather than public clouds and do have some remnants of old legacy systems. They also have a need for automation but more leaning towards enterprise IMS vendors who are all rebranding their products as “IAC” ready.

Many of these tools allow some form of automation but not the full needs of an IAC platform such as version management, change management with automatic regression tests and full integration of various tools in the enterprise. My suggestion to you is to go for a pragmatic combination of current enterprise tools and new age tools to support your large customers.

Infrastructure automation tools makes it possible to carry out actions repeatedly, across a large number of clusters, various levels such as within DMZ, outside DMZ, selected clusters etc.  Infrastructure as code uses techniques, practices, and tools from software development to ensure those actions are thoroughly tested before being applied to business critical systems. Key benefit is inclusion of Security SW deployment and testing as Built-in BAU rather than an additional extra task to be grudgingly done by some other group.

These tools should also work with GIT kind of Opensource configuration management tools. They should allow for quick validations at small cluster level for immediate testing and feedback and rollback if necessary. History of every change for auditing and trouble shooting is a must.

What are the advantages?

Less Cost, More Speed and reduced Risk by removing manual errors and using best practices of SW.

Will also have a softer benefit of faster Devops implementation across the enterprise.

What Next?  What we have just started and you are talking about Next?

How can anything in this day & age not have an AI flavor?

Current tools and practices are what we call first two legs of IAC i.e. Declarative and Imperative. But we need to move in to Environment Aware or Intelligent IAC where IAC can suggest finer changes to the code and (make the changes itself!) by looking at your production e2e environment, mix of all application dependencies and relationships.

Implementation Challenges

Finally, 3 advices for all of you in the Infrastructure Management Services industry.

Learn, Learn and Learn.

Change Management experts say they are 4 stages in dealing with any Big and dramatic changes. (IAC basically takes away the floor you are standing on !.)

1. Ignoring and denial

2. Anger and upset

3. Fighting and resistance

4. Adapting and acceptance

Many of you are in stages 1 to 3 with very few reaching early stage 4.  However, do not worry.  Your first reaction and actions are all valid normal human reactions written down in Change Management books decades ago. However, we need to quickly pass the stages 1 to 3 and move in to 4 with full focus, discipline and rigor. 

This Train (new age IMS Business) is stopping at Platform 4 only for a few minutes. We cannot be endlessly keep waiting in Platform 1,2,3; we need to get past platforms 1,2 and 3 and move to platform 4 and catch the train. Otherwise competition who may be already in platform 4 will catch the train and leave us behind.

The only way to break the Fear of Automation/IAC is to have faith in automation, but believe in the idiom ”In God we Trust but we test everything else”. Start with a small cluster of servers, storage and applications. Let the automation run hands-free with full configuration management, testing and reversal capabilities for a few hours starting with lean periods. Keep adding more and more as Faith becomes Logic.

In conclusion, as we move from “ Physical Iron” to “virtual Machines, we can treat infra as Software and can manage it with Code and use all the good software development principles to truly benefit the enterprise.

Best wishes in this great exciting journey towards IAC.

More later,

L Ravichandran