A Year Without the AWS Console

The title and image seem like a group therapy: I'm Lorenzo and today is one year without the AWS Console :)

TL;DR

• I use only Terraform for all infrastructure configuration
• I enter to AWS Console only for CloudWatch Dashboard and Cloud Cost Explorer
• Why my DevOps life has completely changed

In today's fast-paced world, technological advancements have revolutionized various aspects of our lives. One such innovation is Terraform, a powerful infrastructure automation tool that has transformed the way we manage and deploy our digital infrastructure. In this article, I will share my journey and experiences with Terraform, highlighting the profound impact it has had on my life.

Learning curve

I start from the learning curve because it is the initial stumbling block I had in the first approach.

My DevOps background involved manual configuration. Switching to Terraform was not easy.

Here are some of the reasons:

1. Complex Syntax: Terraform uses its language called HashiCorp Configuration Language (HCL). Although it's designed to be human-readable, it's not as common or well-known as languages like Python, JavaScript, or YAML, which are often used in other IaC tools. This means that new users often need to spend significant time learning HCL before they can effectively use Terraform.
2. State Management: Terraform keeps a state of your infrastructure to prevent conflicts and inconsistencies. State management can become complex, especially in large environments or when multiple team members are working concurrently. New users often struggle with managing state files, understanding how changes affect the current state, and resolving conflicts when they occur.
3. Debugging and Error Messages: While Terraform's error messages have been improving, they can sometimes be cryptic or non-descriptive, especially for beginners. This makes troubleshooting and debugging difficult.
4. Modular Structure: Understanding how to correctly modularize Terraform code can be a challenge. It's important for code reusability and managing complex infrastructures, but it requires a good grasp of Terraform and infrastructure concepts.
5. Version Compatibility: Terraform providers are updated independently of Terraform itself. These updates can sometimes introduce breaking changes, requiring you to update your code. This makes it crucial to understand versioning and how to manage different versions of providers and Terraform itself.
6. Infrastructure Complexity: As with any IaC tool, the complexity of the infrastructure being managed can compound the learning curve. Understanding how to model and manage complex infrastructures using Terraform can be daunting for newcomers, especially those without a strong background in infrastructure and networking concepts.

Apply the approach "Learn by doing"

Learn by doing it's for me like a mantra. I use this approach in my professional life. In this particular case, I use this approach to recreate all of my company's AWS infrastructure while simultaneously learning the Configuration Language.

I don't know if this is the best approach. My side helped me like on other occasions to know being able to test directly, making mistakes in many cases but understanding the error faster and how to solve it.

The Power of Learning by doing

The power of "Learn by Doing" refers to the effectiveness of learning through practical experience and active engagement with tasks or projects. Instead of passively receiving information or instructions, this approach encourages individuals to actively participate, experiment, and learn from their actions and outcomes.

Implementing Learn by Doing with Terraform

This is my step-by-step guide

1. Gain foundational knowledge: Start by familiarizing yourself with Terraform's key concepts, syntax, and resource types. This can be done through online tutorials, documentation, or educational resources.
2. Set up a development environment: Install Terraform on your local machine and configure it to connect with your preferred cloud provider, such as AWS, Azure, or Google Cloud Platform. This step involves setting up authentication credentials and configuring access permissions.
3. Identify a project: Choose a specific infrastructure project or scenario to work on. It could be creating a virtual machine, setting up a network, or deploying a web application. Select a project that aligns with your learning objectives and provides opportunities for hands-on experimentation.
4. Experiment and iterate: To deepen your understanding, experiment with different configurations, try out advanced features, or explore additional resource types. This iterative process allows you to gain hands-on experience and discover the full capabilities of Terraform.

Lessons from Learn by Doing

By embracing the "Learn by Doing" approach with Terraform, you not only gain proficiency in using the tool but also develop essential skills such as problem-solving, collaboration, and critical thinking. These lessons empower you to tackle complex infrastructure challenges, automate deployments, and contribute to the broader ecosystem of infrastructure automation.

Apply the approach "Divide-et-impera"

The second approach that I apply it's "Divide-et-impera".

This ancient strategy, coined by Julius Caesar, promotes the breakdown of large problems into smaller, more manageable parts. In the context of Terraform, this principle is realized through modular infrastructure development and management.

The Power of Modularization

Modularization, in essence, is a process of dividing a system into separate, self-contained modules. Each module performs a unique function and can be developed, tested, and scaled independently from the others. This division allows for greater efficiency and flexibility, as developers can focus on individual components without disrupting the entire infrastructure.

Terraform promotes modularization through its resources and modules, enabling developers to build infrastructure components independently and then integrate them as necessary. This approach streamlines the development process, facilitates collaboration among team members, and makes the infrastructure more maintainable and scalable.

Implementing Divide et Impera with Terraform Modules

To implement the Divide et Impera principle with Terraform, we turn to its most powerful feature: modules. A Terraform module is a container for multiple resources that are used together. Modules can be used to create lightweight and reusable abstractions, so you can describe your infrastructure in a high-level language that's more about the use case and less about the low-level implementation details.

Consider a scenario where you are building an AWS infrastructure with several similar EC2 instances. Rather than writing code for each instance individually, you could create a module for an EC2 instance and then instantiate it as many times as required. This module could contain not only the instance itself, but also associated resources like EIPs, IAM roles, and security groups.

Module Versioning and Source Control

One of the many advantages of Divide et Impera is that each module can be developed and versioned independently. This means that individual parts of your infrastructure can be upgraded, scaled, or even entirely replaced without disrupting the rest of your system.

Terraform integrates seamlessly with source control systems like Git. By keeping each module in a separate repository, teams can work in parallel, managing their own parts of the infrastructure. As long as the module's interface remains consistent, it can be developed, tested, and deployed independently from other modules.

Lessons from Divide et Impera

Applying the Divide et Impera principle to Terraform can yield significant benefits. It enables rapid development and testing of infrastructure components, provides a way to manage complexity in large projects, and promotes team collaboration. Furthermore, the modular design enhances the scalability of your system and allows for improved fault isolation.

It is essential to remember, however, that while the divide-and-conquer approach can simplify many aspects of infrastructure management, it also requires careful planning and discipline to ensure consistency and compatibility between modules. Always strive for a balance between modular independence and overall system cohesion.

In the next chapter, we will examine best practices for structuring Terraform modules and strategies to manage inter-module dependencies effectively, ensuring the successful application of the Divide et Impera principle in your Terraform projects.

Cloud Cost Optimization

Cloud cost optimization is the process of reducing your overall cloud spend by identifying mismanaged resources, eliminating waste, reserving capacity for higher discounts and right-sizing computing services to scale. The goal is to manage your cloud resources so that you're only paying for what you use and need.

Terraform it's for me the best way to achieve this goal.

1. Resource Standardization: Terraform code can be used to create standard templates for your infrastructure. This ensures that only necessary resources are provisioned, and all resources are optimized for their purpose.
2. Environment Cleanup: With Terraform, it's easy to spin up and tear down environments. This allows you to ensure that you're only paying for resources when you're actually using them. You could, for example, tear down non-production environments outside of business hours.
3. Version Control and Collaboration: Terraform code can be shared and collaborated on just like any other codebase. This means that you can more easily audit your infrastructure and keep track of changes, potentially catching costly mistakes before they're made.
4. Multi-cloud Deployment: Terraform is a cloud-agnostic tool, which means it supports multiple cloud providers. If certain services are cheaper on one cloud platform compared to another, you can use Terraform to manage resources across multiple clouds, taking advantage of the best pricing from each.
5. Scalability: With Terraform, you can manage your infrastructure's scalability, ensuring that you're not over-provisioning resources. It's easy to modify your configuration to match the size of your workload.
6. Reducing manual errors: By using IaC, manual errors that can occur from configuring infrastructure are drastically reduced. This in turn can reduce the costs associated with downtime or service disruption.
7. Tagging all resources: With Terraform, you can tag all resources and you can monitor your cost by tag.

Certification

In January 2023 I obtained the certification Terraform Associate (002)

Was it necessary for the job I was doing? No

Did it help me consolidate my knowledge of Terraform? Yes

Here are a few reasons why a Terraform certification is important:

1. Industry Recognition: A Terraform certification is recognized by industry and can make you stand out among your peers. It signifies your dedication, commitment, and specialized knowledge in infrastructure management.
2. Increased Job Opportunities: Terraform is used by many organizations, big and small. With a certification, your chances of finding job opportunities increase as organizations often prefer certified professionals when hiring.
3. Enhanced Skills: The process of getting certified helps you gain an in-depth understanding of Terraform and its use cases. It increases your proficiency in infrastructure automation, management, and orchestration.
4. Higher Salary: On average, certified professionals earn higher salaries than their non-certified counterparts. A Terraform certification can provide a substantial boost to your earning potential.
5. Future-Proofing: As more organizations move towards automated infrastructure and IaC, knowledge of tools like Terraform will only become more valuable. A Terraform certification is a good way to future-proof your career.
6. Professional Growth: The certification can help you grow professionally, opening up opportunities for roles like cloud engineer, DevOps engineer, or infrastructure architect.
7. Credibility: A Terraform certification provides credibility when interacting with stakeholders, colleagues, or customers. It shows that you have the skills and knowledge to effectively use Terraform in different scenarios.

Remember, however, while a certification can provide many advantages, it's equally important to gain hands-on experience with Terraform. Real-world experience, backed by certification, is a powerful combination that can significantly enhance your career in the tech industry.

Final consideration

After a year of not creating any AWS resources via the console and after two years of learning by doing with Terraform:

• My daily DevOps time decreased by 60% ( In our little team the DevOps activities they are entrusted to me )
• The Cloud Cost decreased by 20% ( Calculating that during these two years we have released new services )