A Career in DevOps

In the past week, I got this question on a couple of occasions about how to make a transition and progress in a career in DevOps. This article is an attempt to address these questions in detail.

What is DevOps?

There are several popular definitions for DevOps in use

• It is a set of tools called DevOps Tools.
• CI-CD is DevOps.
• Breaking the silos of development and operations is DevOps.
• It is not a role or a team. It is a philosophy.

All these are correct to a certain extent. Let's explore these definitions in detail.?

• It is a set of tools called DevOps tools: -?Of course, one cannot implement the principles of DevOps without the tools. However, just implementing a bunch of shiny new tools is not DevOps. And you see that in the numerous failed implementations. DevOps is certainly more than just tools.
• CI-CD is DevOps: -?Yes, CI-CD is at the heart of DevOps, and it might be challenging to see DevOps without it. However, the devil is in the details, and DevOps maturity depends significantly on how mature your CI-CD process is, and CI-CD is just one key component of DevOps.
• Breaking the silos of development and operations is DevOps: -?That is undoubtedly one of the primary goals of DevOps. However, DevOps also aims to break the silos of other teams, like QA, Security, and Systems. And breaking the barriers is just one of the many steps to the final goal of continuously delivering value to customers.
• It is not a role or a team. It is a philosophy: -?True that. However, Patrick Debois, who coined the term DevOps in the October of 2009, admitted that "There never was a grand plan for DevOps as a word." Indeed, the word DevOps has evolved in its meaning and influence. It sure started with the idea of Dev and Ops collaboration. Many aspects of DevOps, like collective ownership and accountability, transparency, etc., are philosophical. However, DevOps implementation requires a specific set of skills and a mindset. People having these skills are called DevOps professionals, and teams doing these implementations are called DevOps teams. So, I beg to differ. DevOps is a role, a team, and more.?

?So, what is DevOps?

?DevOps is the amalgamation of people, tools, and processes primarily focused on automating the software delivery process. DevOps helps you deliver your software in an error-free, secure, reliable, and repeatable way in the shortest possible time with the least amount of human intervention.?

?

Details on the various aspects of DevOps are probably the content for another post.?

Why is DevOps a such a challenging (read rewarding) role?

?The biggest challenges of the DevOps role also make it the most exciting and rewarding if you are up for the challenge. Let's focus on the difficulties first.

• DevOps roles do not have the same definition everywhere: -?A DevOps role has varied interpretations and forms, and no two Devops job requirements are the same. Even when you land a DevOps role, there is often no clear and defined boundary where your part ends. These boundaries are very dynamic, and that is how it should be if you have the right spirit of DevOps.
• Dynamics are unique to every organization: -?No two organizations do Devops the same way. You will see later that the area of influence for Devops is so vast that there is no silver bullet solution to the DevOps problem, and each shop must evolve its DevOps game plan as per its unique situation.?
• A role that is spread wide and deep: -?A DevOps role is not only spread wide when it comes to tools, technologies, and processes but also deep, in these areas. To be a successful DevOps professional, you not only need to be a jack of all trades but also a master of quite a few.
• Tools sprawl: -?Add to all this the number of tools in the market to implement the various aspects of DevOps. And this vast tool sprawl is so dynamic that the landscape changes every few years. Keeping on top of the latest and greatest is a constant endeavor for DevOps professionals.?

And all these challenges make the DevOps role such a rewarding experience. A DevOps professional is at the center of all the action learning new things regularly. You almost always could see your work's impact and are beautifully poised to make a difference. In addition, you collaborate with all the teams and get to pick new skills in some form or the other. You will never have a dull day being a DevOps professional.?

What do DevOps Engineers do?

The short answer is a lot of things. However, as I mentioned earlier, the DevOps role varies significantly, and what a DevOps engineer does can vary significantly from company to company. Also, the DevOps role overlaps with other established positions in the industry like Systems engineers, Operations Engineers, Security Engineers, and even Developers and Testers. Larger organizations have many of these specialized roles separate from DevOps. However, DevOps engineers do get to do these roles at some point. Following is a superset of all the things a DevOps engineer might end up doing.

Applications: -?Ownership of some of these roles is shared with the Development, Testing, Systems, and Operations teams.?

• ?CI-CD: -?Setting up CI-CD pipelines, Integrating the DevOps toolchain, version control, branching, versioning, testing stages of the pipeline, building applications, build promotion across environments, incorporating security tooling to the pipeline, managing environment specific application parameters
• Packaging/Containers: -?Creating managing containers, container orchestration, creating and managing installers, packages, libraries
• Automation: -?A lot of programming focus here. Integrating the DevOps toolchain with custom Code, writing plugins, or automating complex operations.
• Deployments: -?Deploying workloads, Deployment strategies, Application/Web server management, Deployment toolchain, managing environment configurations.
• Artifacts: -?Managing artifact repos and generated artifacts, dependency management, artifact lifecycle management, and artifact security.
• Backend/Middle tier:?- Configuring and managing databases, MQs or similar technologies.

Systems: -?Ownership of some of these roles is shared with the Systems team.

• OS: -?Managing SOEs, OS hardening, patching, and upgrades, managing OS performance?
• Networking:?Networking is a specialized role in big organizations; however, the Networking responsibility falls on the DevOps team in small and medium-sized organizations. This involves network design and implementation, VPC design, Firewall, DNS, Load Balancers, VPNs, and site connectivity.
• Tools Administration: -?This is a fast-dying role with the advent of SAAS tools. It was, however, a very vibrant segment at one time. Remember the ClearCase days when ClearCase admins had Rockstar status. Companies still manage their on-prem tools for various reasons and need people to manage these tools.
• Configuration Management: -?Managing the inventory of all the DevOps platform assets, provisioning de-provisioning servers. Running post configuration on servers and ensuring trackability and traceability of all the changes to all the assets

Security: -?Ownership of some of these roles is shared with the Systems, Security, or Identity and Access Management teams.

• Identity & Access Management: -?Setting up and managing Single Sign-On, Identity and Access Management, Active directories, Authentication, Authorisation, Role-based or Attribute-based access control.?
• Application security: -?Implementing SAST, DAST, SCA, and Container security. Ensuring code quality, vulnerability management, third-party dependency scanning, Open-source compliance, Obfuscations, etc.?
• Secrets Management: -?Managing secrets, certificates, keys, tokens, managing the lifecycle of secrets?
• Infrastructure Security: -?Network security, endpoint security for deployment environments, patching, upgrades, security information and event management, privileged access management, etc.

?Operations: -?Ownership of some of these roles is shared with the Operations and SRE teams.

• Logging and Monitoring: -?Implementing logging solutions, ingest, storage, transformation, and archival of log data, visual representation of data, creating dashboards, setting up triggers, monitors and alarms, application performance management, and user experience management.
• SRE: -?Ensuring health and continuity of workloads, Ensuring KPIs and SLAs are met.
• DevOps support: -?Helping other teams who are consumers of the DevOps tools and processes, ensuring the health of DevOps tools and processes, maintenance, and upgrade of DevOps tools and processes.?

?Data & ML: -?This is a relatively new addition to the DevOps role and is fast emerging. This role overlaps with Data Engineers and Data Scientists.?

• Data Ops: -?Data transformation pipelines, data orchestration, CI-CD for data platform components.?
• ML Ops: -?Managing machine learning model pipelines, managing model lifecycle, CI-CD?

What is the absolute minimum set of skills required for a DevOps role?

• ?Cloud: -?It's challenging to imagine Devops without Cloud these days. Good knowledge of at least one cloud vendor is an absolute must, preferably AWS or Azure. An associate-level certification is desirable.?
• ?Operating System: -?Advance user knowledge of at least one OS, preferably Linux.
• ?Programming language: -?This is a much-discussed question, "Do DevOps engineers need to Code?" The short answer is, "Yes." In the current times when almost every operations aspect is As Code, for example, Infrastructure as Code, Pipeline as Code, Security as Code, Compliance as Code, it would be a challenge if you are not aware of programming constructs. Also, DevOps engineers manage automation and builds. Therefore, good knowledge of at least one programming language will make things easier to handle. If you picked one, it would be Java. Once you have a good grasp of one language, picking up other languages will be easy.
• Version Control: -?Being very well conversant with Git and branching and merging methodologies and workflows like Gitflow is essential for a DevOps engineer.?
• Build tool: -?The primary role of a DevOps engineer is to build and deploy software, so having good knowledge of one build tool is very important. If you pick Java as your language of choice, then Gradle is the first choice, and Maven is a close second. Maybe pick up both. Once you understand a build and release tool well, translating it to other tools would be easy.
• ?Pipeline Orchestrating Tool: -?This is the heart of the CI-CD process, which ties together your entire DevOps toolchain and defines all the stages and promotion criteria of your CI CD pipeline. If you selected one tool, it would be Jenkins. Once you master a tool, it is easy to translate it to other tools. And when I say Jenkins pipeline, I mean Jenkinsfile and pipeline as Code. Manually configured Jenkins jobs are no more relevant.?
• Containers: -?Workloads are increasingly packaged as containers these days, and knowledge of Docker and container orchestration tools, preferably Kubernetes, is?fast becoming a prerequisite for DevOps engineers.?

With these skills, you could probably start the DevOps journey; however, you will have to continuously add to this list of skills and tools to go far on the DevOps road.

A case for the "Full-Stack DevOps Engineers."

As we saw earlier, the DevOps role is quite varied, and it can have a very focused set of responsibilities as well. However, a breed of DevOps engineers has mastered most of the skills listed above, if not all. I choose to call them "Full Stack DevOps Engineers." You could compare them to a swiss army knife in real life. You throw them into any DevOps implementation, and they will deliver. These engineers have the spread and depth of expertise and have exposure to various tools and implementations. There is a clear differentiation between them and other DevOps professionals with a more focused experience. I suspect "Full-stack DevOps engineers" will be a norm in the direction industry is heading. I rue that this term is not much in use in the industry.?

How to pick up DevOps skills?

Real-life projects: -?There is no substitute for learning on a real-life project and implementing requirements and solving problems unique to the DevOps role. However, you cannot just rely on this unless you are working to build an extensive DevSecOps platform. This is primarily because a typical DevOps implementation might have a very focused goal with a defined set of tools. If you would want to progress far in a DevOps career, you would need to supplement your on-project learnings with other sources like the ones listed next.

Certifications: -?If you do not have an opportunity to work on a real-life project, the next best thing is to get a certification. A certification is also a good way to consolidate your knowledge about a topic. It also adds weight to your resume. Preparing for certification gives you the rigor and coverage of topics within a tool or technology. However, if learning is your goal, then just focussing on getting the certificate, using question banks and exam dumps, might not be sufficient. It would help if you studied to excel in the exam, probably preparing you better for a real-life DevOps challenge.?

Online Courses: -?These are a great way to pick up skills; however, you might have to curate the courses you pick. Just picking the most popular course might not be sufficient. Do your research on the technology, list down topics you want to learn, see how hands-on a particular course is, and then choose. Focus on finishing available content rather than procuring more content. Once you get a head start, you will know where next to go. Another tip, most online courses have a much slower speed to accommodate all skill levels. If you are comfortable with the topic, consider doing it at 1.5X speed. And no shame in slowing it down if things get too busy for you.?

YouTube: -?Word of caution here as well. The most popular video is probably not the best. It could be a great starting point. I am astonished at times as to how absolute gems of videos have such low views. There is a lot of great content in conference videos, but again with very few views. This is because YouTube rewards content creators who regularly post content with wider viewership (read basic). Curate your playlists.

How-to progress in a DevOps career??

If you are at the start of a DevOps career or have been in the role for a few years, here is the blueprint of how you can go about acquiring Devops skills that will help you progress in a DevOps career. Again, in the spirit of DevOps, this is not the only way to do it. However, I think this is an excellent way to go about it.

Acquiring skills: -?If I listed the Devops skills in the order of jobs available in the market, it would be the following

1. Applications
2. System
3. Security
4. Operations
5. Data & ML

The details of these skills are mentioned above in the "What do DevOps Engineers do?"?section above.

I would say that apart from applications, all the other skills overlap with IT profiles dedicated to these skills. So, it makes sense that you start to acquire these skills in the same order. I am not suggesting that you don't progress to a new skill unless you are an expert in the previous skill in the order of precedence. This list is just an indication of how you should prioritize your efforts. Your goal should be to have mastery over all these skills. If your job does not give you exposure to these skills, refer to the "How-to pick up DevOps skills?" section above on how to go about picking these other skills.?

Tools exposure: -?As I mentioned earlier, the DevOps tools sprawl can be daunting. However, it is not that tough to tackle. Your goal should be to pick up at least one tool in each skill group and have complete mastery over it. This is paramount since it will give you the base needed for picking other tools fast. Once you have mastered this one tool, pick up other popular tools in that skill group. See how the features map. Maybe do a hands-on implementation of a basic use case. Skim through the documentation and check out some product videos. You would be surprised to see that this would be enough for you to prepare you for a more involved implementation of the tool.?

Building things from scratch: -?The initial few years in your DevOps generally would be about maintaining existing processes or enhancing them. As you develop your DevOps skills, you should be in a position at some point in time when you would be required to architect processes and tools implementations and would be setting up things from scratch. These opportunities help solidify your knowledge as expertise and take them to the next level. It would be best if you were on the lookout for such opportunities.

Big picture: -?As you climb the skills and corporate ladder, it will become vital for you to be able to tie business cases to DevOps implementations. Getting exposure to how DevOps solutions are designed, how tools choices are made, and how factors like budget and resourcing impact these choices would become important and desirable. You should be able to identify bottlenecks and improvement opportunities in existing processes. You would be required to architect complex systems from scratch. You would also be required to have a deep understanding of DevOps concepts, and tools, and you should be able to communicate these concepts to leadership and other teams in relatable terms. You will also have to hire and mentor teams and make staffing decisions. While the business aspects of the role do increase as you climb up the ladder, I strongly feel that being hand on and having relevant technical knowledge is paramount for you to succeed in a DevOps career.?

Making the switch to a DevOps role

?Being a rewarding career in various aspects, it is natural that a DevOps role attracts people from various other functions. I would segregate these profiles into four different types. Following is a strategy for these profiles to make the switch

1. Someone from a non-IT background: -?This is a tough transition and the most time-taking, however not an impossible one. A word of caution here. Gone are the days when a DevOps role was a non-technical one and was an easy gateway to a career in IT. Your strategy should be first to get an understanding of computer science fundamentals like operating systems, networking, programming, scripting, databases, agile, etc. Once you have the foundation in place, pick up all the skills mentioned in the?"What are the absolute minimum set of skills required for a DevOps role?"?section above. This should set you up for a start in a DevOps role.
2. System Engineers: -?This is the most common transition I see, and a logical one too. You are somebody who already has some of the skills needed to be a DevOps engineer. Your focus should be to better understand the application aspects like builds and deploys and packaging and software release. It would be best if you learned to build CI-CD pipelines from scratch and then continue building your DevOps skills in the order mentioned in the?"How-to progress in a DevOps career?"?section above.
3. Developers/Testers: -?Not many developers/testers choose to go this route; however, I suspect this will become more common in the future since DevOps is increasingly evolving into a role that needs a lot of developer skills. This is also the most effortless transition since you have some amount of exposure to the skill sets required to be a DevOps engineer. Where I have seen people struggle with this transition is with adopting the mindset of combined ownership and flexible boundaries. While you continue to pick up the various DevOps skills, spend time understanding the DevOps philosophies and mindset and try to incorporate them into the way you operate.???
4. Grads from an IT background: -?This segment has enormous potential, and I am surprised to see its underutilization. You are someone who already has all the basic IT know-how needed to pick up the required Devops skills. You need to ensure you have all the skills mentioned in the?"What are the absolute minimum set of skills required for a DevOps role?"?and then build up from there.?

My journey in the DevOps world?

?I started my career as a C++ developer in the year 2001. Somehow, I ended up being in projects of all kinds from the onset. From testing to web development to writing installers and maintaining version control tools. One day I was writing device drivers, and the other day I was creating Flash animations. From writing web crawlers to setting up VPN, I did it all. This was when C++ developers refused to work on java projects because it would dilute the technology focus of their resumes. However, my focus was to pick as many tools and technologies as possible. Little did I know that I was preparing myself for a career in DevOps. Eventually, I transitioned from a jack of all trade developer to a build and release engineer role. A transition almost unheard of in those days. DevOps, as a term, would still not be in use for the next eight years of my career. For the first ten years of my career, I was an individual contributor to big teams, being the only one doing what I was doing. After that, I joined a team of fantastic DevOps engineers and spent another ten years forging my DevOps skills in the heat of battle. In the last few years, I have been fortunate enough to get opportunities to build complex DevSecOps platforms from scratch. The learning has not stopped, and the journey continues still today.?