50+ Useful DevOps Tools
Agustin Romano
Director of Cloud Architecture & Engineering at Caylent (We're Hiring!) | AWS Certified x4 | CKA
What is DevOps?
The DevOps methodology, a software and team management approach defined by the portmanteau of Development and Operations, was first coined in 2009 and has since become a buzzword concept in the IT field.
DevOps has come to mean many things to each individual who uses the term as DevOps is not a singularly defined standard, software, or process but more of a culture. Gartner defines DevOps as:
“DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology—especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.”
As you can see from the above definition, DevOps is a multi-faceted approach to the Software Development Life Cycle (SDLC), but its main underlying strength is how it leverages technology and software to streamline this process. So with the right approach to DevOps, notably adopting its philosophies of co-operation and implementing the right tools, your business can increase deployment frequency by a factor of 30 and lead times by a factor of 8000 over traditional methods, according to a CapGemini survey.
The Right Tools for the Job
This list is designed to be as comprehensive as possible. The article comprises both very well established tools for those who are new to the DevOps methodology and those tools that are more recent releases to the market—either way there is bound to be a tool on here that can be an asset for you and your business. For those who already live and breathe DevOps, we hope you find something that will assist you in your growing enterprise.
With such a litany of tools to choose from, there is no “right” answer to what tools you should adopt. No single tool will cover all your needs, and will be deployed across a variety of development and Operational teams, so let’s break down what you need to consider before choosing what tool might work for you.
- Plan and collaborate: Before you even begin the SDLC, your business needs to have a cohesive idea of what tools they’ll need to implement across your teams. There are even DevOps tools that can assist you with this first crucial step.
- Build: Here you want tools that create identically provisioned environments. The last you need is to hear “But it works for me on my computer”
- Automation: This has quickly become a given in DevOps, but automation will always drastically increase production over manual methods.
- Continuous Integration: Tools need to provide constant and immediate feedback, several times a day but not all integrations are implemented equally, will the tool you select be right for the job?
- Deployment: Deployments need to be kept predictable, smooth and reliable with minimal risks, automation will also play a big part in this process.
With all that in mind, I hope this selection of tools will aid you as your business continues to expand into the DevOps lifestyle.
Tools Categories List:
Continuous Integration and Delivery
Infrastructure As Code
1. AWS CloudFormation
AWS CloudFormation is an absolute must if you are currently working, or planning to work, in the AWS Cloud. CloudFormation allows you to model your AWS infrastructure and provision all your AWS resources swiftly and easily. All of this is done within a JSON or YAML template file and the service comes with a variety of automation features ensuring your deployments will be predictable, reliable, and manageable.
Link: https://aws.amazon.com/cloudformation/
2. Azure Resource Manager
Azure Resource Manager (ARM) is Microsoft’s answer to an all encompassing IAC tool. With its ARM templates, described within JSON files, Azure Resource Manager will provision your infrastructure, handle dependencies, and declare multiple resources via a single template.
Link: https://azure.microsoft.com/en-us/features/resource-manager/
3. Google Cloud Deployment Manager
Much like the tools mentioned above, Google Cloud Deployment Manager is Google’s IAC tool for the Google Cloud Platform. This tool utilizes YAML for its config files, and JINJA2 or PYTHON for its templates. Some of its notable features are synchronistic deployment and ‘preview’, allowing you an overhead view of changes before they are committed.
Link: https://cloud.google.com/deployment-manager/
4. Terraform
Terraform is brought to you by HashiCorp, the makers of Vault and Nomad. Terraform is vastly different from the above mentioned tools in that it is not restricted to a specific cloud environment, this comes with increased benefits for tackling complex distributed applications without being tied to a single platform. And much like Google Cloud Deployment Manager, Terraform also has a preview feature.
Link: https://www.terraform.io/
5. Chef
Chef is an ideal choice for those who favor CI/CD. At its heart, Chef utilizes self-described recipes, templates, and cookbooks; a collection of ready-made templates. Cookbooks allow for consistent configuration even as your infrastructure rapidly scales. All of this is wrapped up in a beautiful Ruby-based DSL pie.
Link: https://www.chef.io/products/chef-infra/
6. Ansible
Ansible is your friend when it comes to automating repetitive IT tasks such as configuration management, application deployment, and intra-service orchestration. With no additional custom security infrastructure and no agents, Ansible is easy to deploy and runs on YAML, allowing you to describe your automation in a way which approaches basic English text.
Link: https://www.ansible.com/
7. Puppet
Puppet may be the oldest IAC tool on this list, and with that comes a lot of experience and maturity in its field, and a bustling community behind it. What sets Puppet apart is its approach to IAC setup and automation, as with Puppet, you’ll need to define a declarative state and Puppet will figure out how best to achieve said state.
Link: https://puppet.com/
Continuous Integration and Delivery
8. CircleCI
CircleCI is software tool for comprehensive deployment processes that provides a state of the art platform for integration and delivery to release their code around the globe through building and testing automation.
Link: https://circleci.com/
9. Harness
One of the first Continuous Delivery-as-a-Service platforms, Harness helps deployment teams automate its entire continuous delivery process and provide security when deployments fail.
Link: https://harness.io/
10. Buddy
With a simple UI/UX, Buddy is an intelligent CI/CD tool that greatly lowers the threshold of entry for DevOps.
Link: https://buddy.works/
Development Automation
11. Probot
Probot provides bot framework for app creation that is optimized by GitHub. Probot apps are easy to write, deploy, and share.
Link: https://probot.github.io/
12. AWS Opsworks
AWS Opworks is for those who utilize Chef Automate and Puppet Enterprise in AWS. With AWS Opworks, you can easily automate how your servers are deployed, configured, and managed.
Link: https://aws.amazon.com/opsworks/
13. Relay
Event-driven architectures are certainly not a new idea, but Relay was designed specifically with DevOps in mind. Relay has a staggering amount of integrations and workflows to use out of the box, and will be vital in automating low-value tasks so you can focus on what is more important for your team.
Link: https://relay.sh/
14. CA Automic Workload Automation
Here is a comprehensive guide on everything CA Automic offers when it comes to your plan on automating workloads.
Link: Here
Usability Testing
15. CloudEndure
Provides a real-time, block-level replication, which ensures better data integrity for sub-second Recovery Point Objective for databases and all applications. The replication algorithm does not affect system performance and does not even require a system reboot.
Link: https://www.cloudendure.com/
Database and Big Data
16. MySQL
MySQL is a relatively easy to use database and is used to store large amounts of information that is a stable, reliable, and powerful solution with advanced features. It has been used by industry titans like Facebook, Paypal, and Google.
Link: https://www.mysql.com/
17. MariaDB
MariaDB is an open-source database server which is created by the developers of MySQL. Some of its notable users are Wikipedia, WordPress.com and Google. It is a good choice for a fast, scalable, and robust server.
Link: https://mariadb.org/
18. Liquibase
Liquibase is an open-source tool for databases which handles changes and deployment management. It also helps teams track database versioning, database schema deployment, and logical changes.
Link: https://www.liquibase.org/
19 .Datadog
Datadog lets you collect all the unused metadata that makes your programs slow such as slow database queries, thrown exceptions, unmanaged error logs and cache misses, and growing upstream services. With Datadog, all these events, service states, and metrics are collected in one place and a handy visual graphic representation is created.
Link: https://www.datadoghq.com/
20. DigitalOcean
DigitalOcean is a rapidly growing cloud hosting provider. In seconds, it can deploy a Linux based Virtual Machine?—known as a ‘Droplet’?—for you. Digital Ocean boasts high reliability with an uptime percentage of 99.99% and average load times of 219 ms.
Link: https://www.digitalocean.com/
21. M3
Brought to you by Uber Engineering, M3 houses large scale metrics and has been in use by Uber for several years after they realized that existing open source ideas did not meet their criteria. M3 was designed from the ground up to maximize every aspect of the metrics pipeline whilst minimizing its hardware footprint.
Link: https://eng.uber.com/m3/
22. Looker
Looker, now part of Google Cloud, is a highly adaptable Business Intelligence and data analytics platform that seamlessly integrates with Redshift, Snowflake, BigQuery, and over 50 SQL dialects. With Looker, you’ll be granted unprecedented insight into your data.
Link: https://looker.com/
23. Apache Hadoop
Hadoop is designed from the ground up to be easily scalable, its framework allows large data sets to be distributed across a single server or thousands of computers. It also boasts a library designed to implement computation and storage on a local level.
Link: https://hadoop.apache.org/
24. HPCC Systems
HPCC Systems leverages two decades of experience in the data industry to bring you its free and open-source end-to-end data lake platform.
Link: https://hpccsystems.com/
25. BigQuery
Brought to you by Google, BigQuery is the search engine’s answer to bringing serverless, cost-effective, and scalable data warehouses to the masses.
Link: https://cloud.google.com/bigquery
26. Apache Cassandra
Cassandra is the go-to tool when it comes to critical data, with its proven fault tolerance and linear scalability, Cassandra ensures your database will always maintain a high level of scalability and availability.
Link: https://cassandra.apache.org/
27. MongoDB
MongoDB takes a unique approach to storing data by storing it in JSON-like documents, this creates a system that is incredibly flexible, scalable, and dynamic. MongoDB believes this is the ideal way to store data, have a look for yourself, and see if you agree.
Link: https://www.mongodb.com/
28. Qlik
With Qlik’s QlikSense and QlikView, your raw data will be highly actionable, Qlik utilizes its own end-to-end approach to data integration and analytics to maximize turning your data into insights in which your business can grow from.
Link: https://www.qlik.com/us/
29. Sisense
Sisense is a driving force behind the building and deployment of analytic apps. The Sisense data and analytics platform offers an agile Business Intelligence system aimed at turning simple data into powerful analytic tools.
Link: https://www.sisense.com/
30. Talend
First coming onto the scene in 2005, Talend was the first open-source commercial software vendor of data integration software and is still a leading contender in its field.
Link: https://www.talend.com/
Monitoring
31. Sensu
Sensu offers comprehensive multi-cloud monitoring, from a single server to the cloud. Sensu integrates with the platform you’re already using and offer strong security with Sensu SSO providers.
Link: https://sensu.io/
32. SolarWinds Server and Application Monitor
Solarwinds introduces its own server and application monitor, a comprehensive tool with an impressive list of features.
Link: https://www.solarwinds.com/server-application-monitor
Testing
33. Vegeta
An adaptable HTTP load testing tool that can be used as a command-line utility or a library.
Link: https://github.com/tsenart/vegeta
the smart Data Testing solution
34. QuerySurge
QuerySurge’s main appeal is its focus on automating the validation and testing of your data. Some notable features are its cross-platform support, built in integration with most Data Integration solutions, and BI testing.
Link: https://www.querysurge.com/
Security
35. Krypton
Krypton is a U2F authenticator mobile app with un-phishable encrypted architecture. All data encryption keys are stored in the local device storage to avoid unwanted external access.
Link: https://krypt.co/
36. Mysterium.network
Mysterium is a network of nodes that provides privacy and security to Mysterium end users. This node image is created for embedded system hardware such as Raspberry Pi.
Link: https://mysterium.network/
37. Okta
Okta is an identity management service. With Okta, you can manage all employees access in one place, Okta features include: Single Sign-On (SSO), AD and LDAP integration, Multi-Factor Authentication (MFA). All this is brought together in the Okta Integration Network which provides multiple integration options.
Link: https://www.okta.com/
38. Palo Alto Networks
A company which offers cloud security services. These services include access security for mobile users from the branch offices. Data protection, governance, compliance, security monitoring, compliance validation and cloud storage security.
Link: https://www.paloaltonetworks.com/cloud-security
39. Smallstep SSH
Smallstep introduces Single Sign-On SSH (SSO SSH), An easy to use system that prevents you from having to gather, ship, and rotate SSH keys every day. All SSH and sudo access can be managed via a single admin panel.
Link: https://smallstep.com/sso-ssh/
Helpful CLI Tools
40. Awless
A command-line Interface (CLI) that aims to help developers manage Amazon Web Services. Awless syncs transparently to a representational local graph of a cloud resource and their relationship.
Link: https://github.com/wallix/awless
42. Snyk-CLI
Snyl CLI will help you find, and more importantly fix, vulnerabilities in your dependencies?—both in an ad hoc network or your CI system.
Link: https://support.snyk.io/hc/en-us/categories/360000456217-Snyk-CLI
43. Daytona
Daytona assists you by being a more streamlined version of the Vault client CLI with special focus on automating authentication and secrets fetching.
Link: https://github.com/cruise-automation/daytona
Development
44. Bitbucket
Bitbucket is built for professional teams where the planning of projects, collaboration on code, testing, and deployment can happen in one place with both free to paid subscription options.
Link: https://bitbucket.org/product
45. Confluence
Confluence is a software tool perfect for project planning, meeting notes, marketing planning and blog posting. Create an open and accessible workspace for your business?—perfect for teams and companies.
Link: https://www.atlassian.com/software/confluence
46. Frame.ai
Frame takes a unique stance on your customer relations and helps identify the ‘why?’ behind your customer outcomes through its continuous monitoring features.
Link: https://frame.ai/
47. Grit
Git helps programmers/developers to store, transfer, share, and copy commits to destination repositories from source repositories.
Link: https://github.com/grailbio/grit
48. JIRA
Jira is a software development tool which helps developers in capturing, assigning and setting priorities in an intended task. It allows the developer to manage the whole system development process making sure that every task is completed.
Link: https://www.atlassian.com/software/jira
49. EditorConfig
EditorConfig helps developers who are working in large groups having different text editors or IDE in maintaining consistent coding styles.
Link: https://editorconfig.org/
50. Tilix
Tilix is a terminal emulator for Linux that follows gnome human interface guidelines. It has the ability to create multiple panes that can be synchronized.
Link: https://gnunn1.github.io/tilix-web/
51. Jsonnet
Jsonnet is an easy to understand extension of JSON. The tool introduces a variety of features such as duplication elimination, integration with custom/existing applications, and can generate other formats including JSON, INI, and YAML among others.
Link: https://jsonnet.org/
52. Hazelcast
Hazelcast provides an in-memory caching solution that delivers innovative, low-latency, data-centric applications. It can accommodate real-time processing of any application with parallel processing multi-threaded architecture.
Link: https://hazelcast.com/
Visualization
53. Cloudcraft
Cloudcraft is a visual designer that helps developers to create architecture diagrams in minutes with smart components optimized for AWS. The tool eliminates the need to develop manual static documentation and specs.
Link: https://cloudcraft.co/
54. DbSchema
With support for SQL and NoSQL databases, DbSchema offers a great and easy to use visual database designer for you to manage your schema.
Link: https://dbschema.com/
Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.
References:
Capgemini.com. 2020. [online] Available at: <https://www.capgemini.com/de-de/wp-content/uploads/sites/5/2016/03/devops-the-future-of-application-lifecycle-automation.pdf> [Accessed 16 July 2020].
Gartner. 2020. Devops. [online] Available at: <https://www.gartner.com/en/information-technology/glossary/devops> [Accessed 16 July 2020].