Smart DevOps Implementation

DevOps is an IT mindset that encourages communication, collaboration, integration and automation among 'software developers' & 'IT operations' in order to improve the speed and quality of delivering software. DevOps culture is characterized by increased collaboration, decreasing silos, shared responsibility, autonomous teams, improving quality, valuing feedback and increasing automation. To achieve this, Dev and Ops must share the responsibility for maintaining the system that runs the software and develop the software to run on the system with increased quality. DevOps teams focus on standardizing development environments and automating delivery processes to improve delivery predictability, efficiency, security and maintainability. 

Automation is crucial for DevOps implementation. The fundamental principle of DevOps is "automate everything", from code commit to production release. DevOps, a software development approach involves continuous integration, continuous deployment, continuous testing & continuous delivery. There are different activities that are performed to achieve this, viz., Version Control, Code Builds, Code Scans, Unit Tests, Code Deployments, Automated Tests, Production Releases, Measure & Validate. A host of different tools, which are integrated together work cohesively to implement these activities continuously & in a sequential manner. These activities performed in a particular order of sequence forms a CI-CD pipeline. These activities forms the different phases of a pipeline. 

A DevOps Platform is an integrated set of these tools working together, helping the team to implement DevOps. There is no one such tool that suffices different project needs & automates all the stages, hence there are different tools that build up the DevOps platform together. An enterprise DevOps platform, which supports multiple apps built on different platforms using varied technologies, is difficult to development & maintain. Most enterprise DevOps teams fail in maintaining an 'always-on' DevOps platform. Issues & outages on the DevOps platform creates chaos & adds to delays in deployments & ultimately resulting in delayed releases.

Definitely there is a focused effort required in maintaining the DevOps platform which is proactive in dealing with incidents & outages. Following are the practices that helps in getting closer towards maintaining a DevOps platform proactively.

Automate everything: There are multiple stages of activities that form a CI-CD pipeline, viz., Code commits, code builds, code analysis & code scans, maintaining binary repository, infra & environment provisioning, code deployments, quality & testing, releases etc. The key to success is "automate everything". Integrate & automate all the stages of the pipeline. There should no manual activity involved in the pipeline. Automation doesn't only bring in efficiency in doing manual activities repetitively, but also adds predictability in the tasks being automated. 

HA & Resiliency: For better performance of the DevOps platform and for better user experience, always maintain an Resilient HA instance of the DevOps platform. Also develop the DR instance of the platform. Practice fail-over & fall-back testing atleast once a year. 

Control DevOps Platform: Developing a DevOps platform takes a huge effort in integrating so many different tools & making them work together cohesively. Once this platform is setup & tested, Changes to this platform should be controlled. The best practice is to maintain a production-like test environment for the DevOps platform. Any new changes, modifications, configuration changes are first tested on the test environment before making the same on the production environment. 

Monitoring (Verify & Validate): Monitoring the DevOps platform continuously is highly important to make it smart & pro-active. Following are the different levels of monitoring that can be setup to have a better visibility & control over the DevOps platform. Only with right level of monitoring, meaningful alerts can be setup which enables DevOps teams to be pro-active in maintaining the platform.

• Monitoring the Infrastructure: The infrastructure that hosts an application environment would be made up of multiple components: servers, storage devices, load balancer etc. Checking the health of these devices is the most basic requirement of monitoring. 
• Monitoring the DevOps Platform: An application would typically be built using multiple third-party tools such as databases, data repositories, full-text search engines (ElasticSearch), BigData platforms, messaging systems and BI and reporting tools. Checking the health of these application components is ensuring the devops platform is healthy. 
• Monitoring the DevOps tools & application: Having a healthy infrastructure and platform is not good enough for an application to function correctly. Buggy code from a recent deployment or third-party component issues or incompatible changes with external systems can cause application failures. Application & corresponding configuration level checks should be implemented to detect such issues.
• Monitoring Business: The applications run in production to meet certain business goals. It is quite possible that an application that runs flawlessly on a healthy infrastructure might still not be meeting its business goals. It is important to provide that feedback to the business at the earliest to take corrective actions required. 

Log Aggregation: Using the aggregated and indexed logs, it is possible to detect issues that would have gone unnoticed earlier. Alerts can be setup based the info available in the indexed log data. Tools like Logstash, Loggly and Splunk are handy for this. Using APIs provided by these tools, the alerting could actually be integrated with the main monitoring platform.

Monitor the Monitoring: It is important to make sure that the monitoring infrastructure used for monitoring the DevOps platform, itself is up and running. Monitor the monitoring infrastructure. There is a catch though. Disabling alerting during a deployment and forgetting about enabling it later is one of the common oversights I have seen in operations. Such missteps are hard to monitor and only improvement in deployment process could address such issues. Hence continuous process improvement is very important.

Metrics, Continuous feedback & Improvement: It is important for the DevOps teams to define goals & have necessary metrics to measure the improvements. It is equally important for the organizations to define necessary metrics to measure the value proposition they derive in investing in the DevOps team and whether their investments are reaping required benefits. These metrics are used to define the road-map for continuous process improvement. There should be a mechanism for different teams & stakeholders to share their feedback, which should in-turn develop into a plan for improvements. Do not piggyback on the common generic measurements provided by the tools you are using, rather define some focused metrics that makes sense to your organization & suffices your business needs.

Transparency & Collaboration: The goal of DevOps is for the dev & ops teams to work cohesively & collaboratively. DevOps demands different teams working in silos to work together as a single team and this is possible only thru transparency & collaboration. Support from the leadership & management team plays a key role here. 

Innovation, Self-Service & Self-Healing automation: Key success factor for DevOps implementation is the level of automation achieved. Focus towards smart automation is required. The automation should be self-service & self-healing in nature. There should be no manual intervention required for the automation to run. Encourage the teams to Innovate and implement latest tools, technologies to make your automation, smarter and faster & self-dependable.