PROJECT COLLABORATION USING REMOTE STATES

Welcome to Advanced Concepts of Terraforming!!

Hey, everyone!

I hope you all are enjoying this series and finding interesting concepts in a simplified manner presenting at your doorstep, haha!

(That sounds like the selling of good products, but it's not like that!)

This is the effort of a cumulative, and we are learning, growing together and sharing things collaboratively, hence contributing to our community at large.

With this thought, I would like to bring the crucial concept to your mind.

And

Yes, it's a very important concept in a real-world production environment.

I hope you will not miss this!

It's like a project on which two teams are working, and they want to collaborate safely.

That's it.

Let's get started!

?? CROSS-PROJECT COLLABORATION USING REMOTE STATES

First of all, I would like to tell you that we have already discussed in a comprehensive way how to establish and work in remote states.

I hope you have gone through this; if not, then you must visit my previous articles or posts where we have covered this in a deeper way.

If you are well aware of it, then read this article carefully!

?? Basics of Terraform Remote State

The terraform_remote_state data source retrieves the root module output values from some other Terraform configuration, using the latest state management from the remote backend.

Let's understand the below architecture diagram.

As you can see from the above, we have two projects: public IPs and security groups.

Now let's suppose both the projects are handled by each different team.

Like

• Public IP << Networking Team
• Security Group << Security Team

Taking Project 1,

Here, public IP is storing its state in a remote state, in our case the S3 backend, and hence due to this, all of our data is stored as part of the S3 bucket.

Here we are talking about the data as the output values stored in an S3 bucket.

Taking Project 2,

Security groups are being created. Security groups are typically used to control inbound and outbound traffic to resources like servers or virtual machines.

These security groups need to list the IP addresses that are part of the output values mentioned earlier.

This implies that the security group rules may depend on specific IP addresses or ranges.

The security group configuration might require fetching IP addresses from the S3 bucket, suggesting that the security groups' rules might be dynamically updated based on the data stored in the S3 bucket.

With this scenario, we have a clear picture in our mind that there is an interdependency between the two projects.

The security group configurations depend on data from the S3 bucket, which is managed by the Networking Team in Project 1.

So,

This means that the Security Team must coordinate with the Networking Team to ensure that the security group rules accurately reflect the state of the public IPs.

?? How do we connect Project 2 (Security Group) with the S3 bucket?

We achieve this through terraform_remote_state.

terraform_remote_state has the ability to connect to a specific state file of a different project, fetch some data from that state file and reference it as a part of the current project.

And this is what we need, as seen from the above architecture for our scenarios.

Let's implement this ideal approach.

?? IMPLEMENTING OF THE ABOVE ARCHITECTURE

For the security group, we need a CIDR block as a rule, and for that, we are fetching the values from project 1 (state file).

therefore,

We can clearly see from the code below that we have a resource of aws_eip that is created, and the output of that value is referenced and stored in the remote state S3 backend.

This is exactly what we have seen in our architecture of project 1 above.

Now in the project 2, we have a "terraform_remote_state" in the data block where we have specified the details about the backend configuration within it, which has the exact path of the S3 bucket and key file.

From the below code, we can easily see that we are fetching the output values from the different projects, and it is referred to in the security group resource. It is all possible using "terraform_remote_state", which is collaborating both projects.

From the below code, in sg resource, we have

"data.terraform_remote_state.eip.outputs.eip_addr" in cidr_blocks

I know that you may be facing difficulty in understanding the components and values used here, but don't worry,

I'm attaching my GitHub link for Day 33 which has the code and you will understand each and every code line and the value associated with it.

You should refer to it for better understanding.

I hope you have understood the concept in a very easy way.

?? SUMMARY

Let me explain what's happening, and it's not as complicated as it may seem.

Step 1:

In our first setup, we have a configuration file where we've created a special resource called EIP. This resource provides us with an important IP address. We also want to share this IP address with others, so we've created an output block that points to it. All of this configuration information is securely stored in a remote state, which is like a special digital safe, and we keep this safe in an S3 bucket.

That's all there is to it for step 1.

Step 2:

Now, in our second setup (a different project or configuration), we have a data_source block. In this block, we use something called terraform_remote_state to get data from our first setup. We tell it exactly where to find the information we stored in that digital safe (S3 bucket). But here's the interesting part: In this second setup, we have another file that has some special rules (like instructions) about who can use what. These rules reference the important IP address from our first setup.

So, in simple terms, we're just getting some information from our first setup (like borrowing a book from a library) to use in our second setup.

That's it, and that's all there is to it!

NOW,

I'm damn sure that you all have understood this concept, architecture, and real use case, and it's workings in a very easy way.

And

I believe that you will be trying this by yourself in your environment.

?? Why are we doing this?

To facilitate smooth cross-collaboration between teams working on their independent projects.

Now, consider this:

If the Project 1 team updated the new public IP address.

What will you do?

Don't worry; take it easy-peasy.

You also have to update, but you just have to apply the Terraform command in your SG configuration file to fetch the new updated output value.

terraform apply        

?? CONCLUSION

In a nutshell, what we've discussed here is the idea of teams working on different parts of a big project and needing to share important information with each other.

We use a tool called Terraform to help with this.

I HOPE YOU HAVE LEARNED A LOT!!

Let me know if there’s anything else I can assist you with!

signing off!

Author: Sahdev Grover

Architecture Diagrams Credit: Zeal Vora

Follow Sahdev Grover for more exciting and informative content delivered right to your doorstep!