The Dummies Guide to DevOps

What is DevOps?

DevOps is a set of practices that works to automate and integrate the processes between software development and IT teams, so they can build, test, and release software faster and more reliably. The term DevOps was formed by combining the words “development” and “operations” and signifies a cultural shift that bridges the gap between development and operation teams, which historically functioned in siloes.

It's a firm handshake between development and operations that emphasizes a shift in mindset, better collaboration, and tighter integration. It unites agile, git, continuous delivery, automation, and much more, to help development and operations teams be more efficient, innovate faster, and deliver higher value to businesses and customers.

DevOps is a culture, a movement, a philosophy!

Challenges Solved by DevOps

Prior to DevOps application development, teams oversaw gathering business requirements for a software program and writing code. Then a separate QA team tests the program in an isolated development environment, if requirements were met, and releases the code for operations to deploy. The deployment teams are further fragmented into siloed groups like networking and database. Each time a software program is “thrown over the wall” to an independent team it adds bottlenecks. The problem with this paradigm is that when the teams work separately:

• Dev is often unaware of QA and Ops roadblocks that prevent the program from working as anticipated.
• QA and Ops are typically working across many features and have little context of the business purpose and value of the software.
• Each group has opposing goals that can lead to inefficiency and finger pointing when something goes wrong.

DevOps addresses these challenges by establishing collaborative cross-functional teams that share responsibility for maintaining the system that runs the software and preparing the software to run on that system with increased quality feedback and automation issues.

A Common Pre-DevOps Scenario

The Dev team that has a goal to ship as many features as possible, kicks a new release “over the wall” to QA. Then the tester’s goal is to find as many bugs as possible. When the testers bring their findings to Dev, the developers become defensive and blame the testers that are testing the environment for the bugs. The testers respond that it isn’t their testing environment, but the developer’s code that is the problem.

Eventually the issues get worked out and QA kicks the debugged new release “over the wall” to Ops. The Ops team’s goal is to limit changes to their system, but they apprehensively release the code and the system crashes. The finger pointing resumes. 

Ops says that Dev provided them faulty artifacts. Dev says everything worked fine in the test environment. The fire drills begin to debug the system and get production stable. The production environment isn’t Dev’s and QA’s responsibility, so they keep hands off while Ops spends all night fixing the production issues.

Benefits of DevOps

Collaboration and Trust

The number one factor of a DevOps team performing well is collaboration. Building a culture of shared responsibility, transparency, and faster feedback is the foundation of every high performing DevOps team. Teams that work in silos often don't adhere to the “systems thinking” DevOps espouses. “Systems thinking” is being aware of how your actions not only affect your team, but all the other teams involved in the release process. Lack of visibility and shared goals means lack of dependency planning, misaligned priorities, finger pointing, and “not our problem” mentality, resulting in slower velocity and substandard quality. DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between development and operations.

Faster time to market

Speed is everything. Teams that practice DevOps release deliverables more frequently, with higher quality and stability. A lack of automated test and review cycles slow the release to production, while poor incident response time kills velocity and team confidence. Disparate tools and processes increase operating costs, lead to context switching, and can slow down momentum. Yet with tools that drive automation and new processes, teams can increase productivity and release more frequently with fewer hiccups.

Accelerated time to resolution

The team with the fastest feedback loop is the team that thrives. Full transparency and seamless communication enable DevOps teams to minimize downtime and resolve issues faster. If critical issues aren't resolved quickly, customer satisfaction tanks. Key issues slip through the cracks in the absence of open communication, resulting in increased tension and frustration among teams. Open communication helps development and operations teams swarm on issues, fix incidents, and unblock the release pipeline faster.

Better manage unplanned work

Unplanned work is a reality that every team faces–a reality that most often impacts team productivity. With established processes and clear prioritization, development and operations teams can better manage unplanned work while continuing to focus on planned work.Transitioning and prioritizing unplanned work across different teams and systems is inefficient and distracts from work at hand. However, through raised visibility and proactive retrospection, teams can better anticipate and share unplanned work.

A Common Post-DevOps Scenario

The software team meets prior to starting a new software project. The team includes developers, testers, operations and support professionals. This team plans how to create working software that is ready for deployment.

Each day new code is deployed as the developers complete it. Automated testing ensures the code is ready to be deployed. After the code passes all the automated testing it is deployed to a small number of users. The new code is monitored for a short period to ensure there are no unforeseen problems and it is stable. The new code is then proliferated to the remaining users once the monitoring shows that it is stable. Many, if not all, of the steps after planning and development are done with no human intervention.

The CALMS Framework for DevOps

CALMS is a framework that assesses a company's ability to adopt DevOps processes, as well as a way of measuring success during a DevOps transformation. The acronym was coined by Jez Humble, co-author of “The DevOps Handbook,” and stands for:

Culture

DevOps isn’t simply a process, or a different approach to development -- it’s a culture change. And, a major part of a DevOps culture is collaboration.

All the tooling and automation in the world are useless unless development and IT/Ops professionals work together. Because DevOps doesn’t solve tooling problems. It solves human problems.

Think of DevOps as an evolution of agile teams - the difference is now operations is by default included. Forming project or product-oriented teams to replace function-based teams is a step in the right direction. Include development, QA, product management, design, operations, project management, and any other skill set the project requires. Rather than having one team do everything or hiring “DevOps professionals”, it’s more important to have project-based teams that can seamlessly work together. 

Automation

Automation helps eliminate repetitive manual work, yields repeatable processes, and creates reliable systems.

Build, test, deploy, and provisioning automation are typical starting points for teams who don’t have them in place already. And hey: what better reason for developers, testers, and operators to work together than building systems to benefit everyone?

Teams new to automation usually start with continuous delivery: the practice of running each code change through a gauntlet of automated tests -- often facilitated by cloud-based infrastructure -- then packaging up builds and promoting them to production using automated deployments.

Why? Computers execute tests more rigorously and faithfully than humans. These tests catch bugs and security flaws sooner. And automated deployments alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.

Another major contribution of DevOps is “configuration as code.” Developers strive to create modular, composable applications because they are more reliable and maintainable. That same thinking can be extended to the infrastructure that hosts them, whether it lives in the cloud or on the company's own network.

“Configuration as code” and “continuous delivery” aren’t the only types of automation seen in the DevOps world, but they’re worth special mention because they help break down the wall between development and operations. And when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “works on my machine!” becomes irrelevant.

Lean

When we hear “lean” in the context of software, we usually think about eliminating low-value activities and moving quickly – being scrappy and agile. Even more relevant for DevOps are the concepts of continuous improvement and embracing failure - which lay the foundation of an experimental mindset.

A DevOps mindset sees opportunities for continuous improvement everywhere. Some are obvious, like holding regular retrospectives so your team’s processes can improve. Others are subtle, like A/B testing different on-boarding approaches for new users of your product.

We have agile development to thank for making continuous improvement a mainstream idea. Early adopters of the agile methodology proved that a simple product in the hands of customers today is more valuable than a perfect product in the hands of customers six months from now. If the product is improved continuously, customers will stick around.

Measurement

It’s hard to prove your continuous improvement efforts actually improve anything without data. Fortunately, there are loads of tools and technologies for measuring performance, like how much time users spend with your product, whether that blog post generated any sales, or how often critical alerts pop up in your logs.

Although you can measure just about anything, that doesn’t mean you have to (or should) measure everything. Take a page from agile development and start with the basics:

• How long did it take to go from development to deployment?
• How often do recurring bugs or failures happen?
• How long does it take to recover after a system failure?
• How many people are using your product right now?
• How many users did you gain / lose this week?

With a solid foundation in place, it’s easier to capture sophisticated metrics around feature usage, customer journeys, and service level agreements (SLAs). The information you get comes in handy when it’s time for road mapping and spec’ing out your next big move.

All this juicy data will help your team make decisions, but it’s even more powerful when shared with other teams – especially teams in other departments. For example, your marketing team wants shiny new features they can sell. But meanwhile, you’re seeing high customer churn because the product is awash in technical debt. Providing user data that supports your road-map – even if it’s light on features and heavy on fixes – makes it easier to build consensus and get buy-in from stakeholders.

Sharing

The long-standing friction between development and operations teams is largely due to a lack of common ground. We believe that sharing responsibility and success goes a long way toward bridging that divide. Developers can win instant goodwill by helping to carry one of operations’ biggest burdens: the pager. DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.

This doesn’t mean that you hire developers and simply expect them to be excellent operators as well. It means that developers and operators pair with each other in each phase of the application’s lifecycle.

Teams that embrace DevOps often have a rotating role whereby developers address issues caught by end users while, at the same, troubleshooting production problems. This person responds to urgent customer-reported issues, creating patches when necessary, and works through the backlog of customer-reported defects. The “developer on support” learns a lot about how the application is used in the wild. And by being highly available to the operations team, the development teams build trust and mutual respect.

As much as we wished that there was a magic wand to transform all teams into high-performing DevOps teams, DevOps transformations require a blend of practices, cultural philosophies, and tools. But like you’ve read, the benefits to breaking down the Development and Operations siloes are well worth it -- increased trust, faster software releases, more reliable deployments, and a better feedback loop between teams and customers.

Implementing DevOps is no small task. Yet given the right framework, effort, and tools, an organization can undergo a DevOps transformation that yields significant benefits.