Part 2 - 10 Commandments for setting up DevOps

DevOps Approach & Steps.

DevOps is all about trust, seamless collaboration and effective communication. It is all about Trust between the “Business & Development” &Trust between “Development and Operations”. Trust among these groups enables the organisation to be more agile and to better manage risk. Bringing trust between teams is the way to go for successful DevOps.

DevOps enables a closer working arrangement between developers and operations personnel; its best practitioners automate where possible and, in the process, they make developers responsible for ensuring their code functions well in an operational environment. Some of the main characteristics of the DevOps philosophy are automated configuration, collaborative development, continuous integration, automated tests and continuous monitoring & feedback and performance tracking. IT agility is attainable through DevOps.

The goal is to add strategic value to a dynamic marketplace. The method of choice builds on agile development and cloud computing with DevOps-centric approaches as well as ongoing evolution and delivery. Business agility will increase as IT agility does. As we all know, consistent configuration, maintainability, audit & accountability makes the system great and good dev & deploy takes you there. Version control provide a great mechanism to manage code, version and collaborate in reliable & predictable ways. Issue trackers that integrate with version control let you easily tie specific changes back to the original reason for the change. Deployment systems give you one stop interfaces to track what got deployed where and when & by whom. Through ticket system, commenting on scripts is very important as these comments link back to the work order number. Configuration management systems is not an automatic event across your entire infrastructure, iterate against it. Make small weekly changes that roll up to quarterly progress.

Migrating what you do today to a more DevOps focused approach isn’t going to happen overnight.

Below are the key steps to build a successful DevOps model.

·        Assess current infrastructure

·        Interviews with all the stake holders from Dev, Business and operations teams

·        Understand pain points, grey areas and difficulties relating to technical & resource aspects

·        Requirement gathering & analysis, present needs & future expectations and budgetary constrains

·        Coaching on DevOps & it’s short team and long-term benefits to the organization. Enablement certainly bring in synergies between various teams and make every team (Dev, Business & Operations) realizes that they have equal accountability / responsibility / stakes in DevOps environment. DevOps is a culture and teams adapts to the culture with great & collaborative thinking and effective communication.

·        Blueprint on the plan, design and role define.

·        Dig deep dive on building a DevOps stack for various functions like; Configure & change management, Release Management, Integration, Testing, deployment, orchestration, monitoring, virtualization and containerization & many more.

·        Induce Agile as DevOps + Agile practice is the way to go for successful DevOps implementation.

Important tasks for DevOps approach

Effective Communication & Collaboration:

The approach is to establish effective communication between developers & operators in knowing & understanding production realities & the application requirements. Always lack of effective communication as a barrier for the accelerated delivery. We have seen mostly development and operations teams work in silos because that's how the executive hierarchy is configured as each team is measured against a different set of metrics: change vs. stability. However, to ensure smooth delivery, it is imperative to establish how the needs of the application will be supported by the final production environment early in the process. The communication might be as lightweight as a face to face meeting where all concerns are discussed, a template to understand what capabilities are needed and provided, or something more detailed like a topology diagram. Regardless of the format, the information must be communicated early.

Collaborative planning & designing:

Key approach is to insist on designing the patterns for platforms and applications jointly by development & operations. One should focus on enabling production-like systems to be leveraged earlier in the development process. Standard application and platform patterns are decided with collaboration between development and operations. Those patterns are then used by developers in an on-demand form when they need to iterate quickly on changes for the application in their own isolated environment. When changes are ready for delivery, the build and deploy process should use the same patterns to deploy and verify each new build. The net is that both development and the continuous delivery process are using production-like patterns to verify their approach far earlier in the delivery process than what you may be used to today.

Version Everything:

The question to ask? Do you version deployment configuration and scripts along with your source code? Often, specialists are already creating deployment scripts today; bash, perl, python, wsadmin, ruby, chef, puppet are all examples of assets which help stand up environments. But for whatever reason, these assets aren't generally handled like the same .java, .jsp, .html or .ddl assets which make up the business logic of the application. This step says that deployment configuration and script assets should be treated like valuable content just like the application source code. The idea is that the automation scripts and configuration files which have been left by the wayside as middleware evolved are brought back into the fold and considered part of the overall solution. Incorporating these artifacts into the same version control systems are valuable for application source code means that you can more easily view and compare changes from prior deployments. If you're deploying the system completely through automation, it may also mean that rolling back a new deployment means simply re-applying the last approved version of the automation.

Automate Everything:

DevOps approach of automating everything is very important but totally different than traditional automation approaches. There is no tool that automates everything. Hence, automating various processes improve agility. In DevOps, automation technologies having file-based persistence which are declarative than imperative, and which are applied again and all over again. The designed patterns are to be based on reusable deployment configuration scripts and both platform & application is to be deployed as an environment in one step. The usual practice of automation is above the platform layer, i.e., to deploy application into an existing platform.

DevOps takes the end goal a step further to stress automation to create the entire solution -- from the platforms all the way to the end user.

DevOps asks the same kind of questions about deployments that agile software asked about builds. The ability to do a build in one step enabled us to perform builds more frequently ensuring that everything compiles. Similarly, being able to do a deployment in one step enables more frequent deployments. If a problem is introduced, you want to find it early and fail fast. With DevOps, operation engineers are no longer act as administrators but more as content creators / infrastructure developers.

Like software developers, infrastructure developers create automation to provision and maintain the infrastructure, platforms, and applications in each stage of your pipeline. We insist on providing production like environments to validate the application as this help to validate the platform, supporting the applications. Hence any new platform updates that you plan to roll out feed into the pipeline just like application builds. If your middleware platform needs to be updated, you would roll that change out first early in a development stage of your pipeline. Just as application artifacts are graduated through the pipeline as they pass their verifications, so are changes to the infrastructure components like your middleware. Treating application and infrastructure components like equal citizens means that there isn't a special exception process for one kind of change or another -- all changes go through the same journey together in harmony.

Track everything:

The most important is to determine how changes are tracked. The best way is to linking bugs or work items directly to the affected changes in version control as it is important to have this level of traceability for identifying what a change was introduced and ideally what related files were touched.

As most of the developers think that real work is done once an application reaches production, it's not! When problems happen in production, there's generally an existing process for tracking incidents. We suggest that whenever incidents are encountered, an incident ticket should be opened and linked directly to defects in either the software or configuration layer, ensuring traceability from end to end. The bug or work item enables you to track when a fix is released to the application or configuration and know when the ticket can be closed. Knowing what the incident looked like in production means you can also beef up test cases so that a regression isn't introduced in the future. It doesn't matter if the root cause is in the business logic or an invalid Data Source configuration, the changes should be treated and managed in the same way. Since bugs are linked to changes, one will know what had to change to fix the incident.

Test Everything

The objective is to ensure that the provisioning process is meeting the expectations of stakeholders including development, quality assurance, and operations. Other stakeholders like performance and security testing may also play a role. Software development did this first with compilation. It ensured that all your source code could be translated into a machine-readable format. Errors were quickly reported as part of the build process. Then agile promoted the notion of incorporating unit tests, static analysis, and other security compliance scans. DevOps expands that notion to also testing the deployed software and its supporting platforms. The recommended approach is to have automated testes to validate the application & platform function and characteristics.

Monitor everything:

Most production systems already have some level of monitoring in use today. Exposing production-like patterns and environments earlier should mean that monitoring capabilities are available to developers. Hence, developers and operators can verify that application code is being written to be easy to monitor. Just like knowing what an incident looks like can help you beef up test cases, knowing what kind of information is (or isn't) coming from an application's monitoring capabilities will help you understand how to provide better feedback in production. Like Steps 7 and 10 are about providing continuous feedback about the function of the application, Step 11 is about validating expectations for performance and reliability through monitoring early in the process.

Dashboard everything:

Most of the times, we see loosely defined, loosely coupled notion of delivery pipeline. Establishing a common understanding of what stages an application must pass through on the way to production ensures that hand offs are predictable and tracked to ensure accountability. Beyond that, providing progress reports about applications and their location through the pipeline -- from inception to happy end user -- is critical to ensure that you're constantly improving the overall delivery response time.

Following this process also means establishing the right metrics for your organisation at a higher level and ensuring that those metrics can be tracked. An example metric might include something like "how long does it take to release a no-change version of my application?" That is, if you don't change a single line of source code, how quickly could you get the current version rolled out into production (starting in development)? Often (surprisingly) this time is non-zero. A lot of it may include manual configuration and intervention which limits how quickly it can be done. Again, focusing on automation from barebones to happy user means that you can increase your delivery velocity with managed risk.

To build successful DevOps, the below 10 Commandments are mandatory

1. Address the IT skills gap

2. Establish an Agile extended DevOps blueprint

3. Invest in right toolchain, Software management approaches and infrastructure

4. Track & Plan everything

5. Automate everything

6. Remove barriers

7. Integrate and modernize legacy systems (protect your investments were made)

8. Test Everything

9. Monitor Everything

10. Continuous performance tracking and feedback as this helps in Continuous Improvement.

Few tips on technology / tooling landscape

Operating Systems – Linux RHEL, CentOS, Ubuntu, Unix (Solaris, AIX, HP/UX), Windows, Mac OS

 Infrastructure as a Service: Amazon Web Services, Rackspace j, Cloud Foundry, Azure, OpenStack

 Virtualization Platforms: VMware, KVM, Xen, Virtual Box, Vagrant

Containerization Tools: LXC, Solaris Containers, Docker

 Linux OS Installation: Kick Start, Cobbler, Fai

 Configuration Management: Puppet Enterprise, Chef, Ansible, CFEngine J, SaltStack, RANCID, Ubuntu Juju

 Test and Build Systems: Jenkins, Maven, Ant, Gradle

 Application Deployment: Capistrano

 Analytics: Telerik Analytic

 Test Automation: Telerik TestStudio

 Application Servers: JBoss, Tomcat, Jetty, Glassfish, WebSphere , WebLogic

 Web Servers: nginx , Apache , IIS

 Queues, Caches, etc. – ActiveMQ, RabbitMQ, Memcache, Varnish, Squid

 Databases: Percona Server, MySQL, PostgreSQL, OpenDAP, MongoDB, Cassandra, Redis, Oracle, MS SQL

 Monitoring, Alerting, and Trending: New Relic , Nagios, Icinga, Graphite, Ganglia, Cacti, PagerDuty J, Sensu

 Logging: Paper Trail, Logstash, Loggly, Splunk, SumoLoguc

 Process Supervisors: Monit, runit, Supervisor, god, Blue Pill, Upstart, Systemd

 Security: Snorby Threat Stack, Tripwire, Snort

 Security Compliance: Puppet Enterprise

 Miscellaneous Tools: Multihost SSH Wrapper, Code Climate, iPerf, Ildpd

PS: Technology and tooling stack would vary as one size does not all. This is has to be validated, discussed, measured to take final call. Also one need to the existing toolchain and accordingly take a decision.

Similarly, the above tools are just examples. They are lot many tools for each function and select what suits best.