Terraform Automation with RDS Snapshots

Introduction:

In today's fast-paced development environments, efficient infrastructure deployment is essential for ensuring smooth operations and rapid iteration cycles. Leveraging automation tools like Terraform alongside AWS RDS snapshots can significantly streamline this process. In this article, we'll explore a scenario where we begin by manually capturing a snapshot of an RDS database instance. We'll then dive into how to automate the creation of a new RDS instance using this snapshot with Terraform, illustrating the power of infrastructure as code for seamless replication.

Let's see the Scenario:

Imagine you're part of a DevOps team responsible for managing the infrastructure for a web application deployed on AWS. Your team needs to set up a staging environment that mirrors the production environment for thorough testing before deploying updates. To achieve this, you decide to leverage Terraform for infrastructure automation and RDS snapshots for replicating the production database.

Snapshot Creation: Before diving into automation, let's first cover the manual process of creating a snapshot for an RDS database instance. Follow these steps:

1. Navigate to the RDS Console: Log in to the AWS Management Console and navigate to the RDS service.
2. Select the Target Database Instance: Identify the RDS database instance for which you want to create a snapshot.
3. Initiate Snapshot Creation: Within the instance details, locate the "Actions" dropdown menu and select "Take snapshot."
4. Specify Snapshot Details: Provide a descriptive name for the snapshot and any additional settings as needed.
5. Confirm and Create: Review the snapshot details and click "Create snapshot" to initiate the process.
6. Monitor Snapshot Progress: Monitor the progress of the snapshot creation in the RDS console until it reaches a "completed" status.

Terraform Automation: Transition to automating the deployment of a new RDS instance using Terraform. Below is a high-level overview of the Terraform configuration:

1. Terraform configuration
2. Define backend and AWS provider
3. Get latest snapshot from RDS DB
4. Create RDS instance from snapshot
5. Define DB Subnet Group

Provider.tf

provider "aws" {
  region = "us-east-1"
}        

The provider block configures the AWS provider for Terraform. It specifies the region where resources will be provisioned. In this case, resources will be provisioned in the US East (N. Virginia) region.

Backend.tf

terraform {
  backend "s3" {
    bucket = "dev-saja"
    key    = "stg-saja/terraform.tfstate"
    region = "us-east-1"
  }
}        

This file configures Terraform to use an S3 backend for storing its state files. State files are used by Terraform to keep track of the current state of your infrastructure. Storing them in an S3 bucket provides durability and allows for collaboration among team members.

rds.tf

# Get latest snapshot from RDS DB
data "aws_db_snapshot" "new_snap_db" {
    most_recent = true
    db_instance_identifier = "myrdsinstance"
}

# Create RDS instance from snapshot
resource "aws_db_instance" "new_db_instance" {
  identifier          = "new-db"
  allocated_storage   = 20
  engine              = "mysql"
  instance_class      = "db.t2.micro"
  snapshot_identifier = data.aws_db_snapshot.new_snap_db.id
  skip_final_snapshot = true
  db_subnet_group_name   = aws_db_subnet_group.my_db_subnet_group.name
}


# Create DB Subnet Group
resource "aws_db_subnet_group" "my_db_subnet_group" {
  name       = "my-db-subnet-group"
  subnet_ids = [aws_subnet.subnet_1.id, aws_subnet.subnet_2.id]
}        

• The data block retrieves information about the latest snapshot of an RDS database. By setting most_recent to true, Terraform ensures it fetches the most recent snapshot. db_instance_identifier specifies the identifier of the RDS instance from which to fetch the snapshot.
• The resource block defines a new RDS instance to be created based on the retrieved snapshot. Parameters such as identifier, allocated_storage, engine, and instance_class specify the properties of the new RDS instance. snapshot_identifier references the snapshot obtained from the data block. skip_final_snapshot ensures that no final snapshot is created when the RDS instance is deleted.

Create DB Subnet Group

• This part of the configuration defines an AWS RDS subnet group resource.
• name specifies the name of the DB subnet group.
• subnet_ids specifies the IDs of the subnets to include in the subnet group. In this case, it references the IDs of two subnets defined elsewhere in the Terraform configuration.

vpc.tf

resource "aws_vpc" "my_vpc" {
  cidr_block = var.vpc_cidr
  enable_dns_support = true
  enable_dns_hostnames = true
}
resource "aws_internet_gateway" "internet_gateway" {
  vpc_id    = aws_vpc.my_vpc.id

  tags      = {
    Name    = "My-VPC"
  }
}        

The aws_vpc resource creates a Virtual Private Cloud (VPC) in AWS. A VPC is a logically isolated section of the AWS Cloud where you can launch AWS resources.

subnet.tf

resource "aws_subnet" "subnet_1" {
  vpc_id            = aws_vpc.my_vpc.id
  cidr_block        = var.subnet_1_cidr
  availability_zone = var.az_1

  tags = {
    Name = "main_subnet1"
  }
}

resource "aws_subnet" "subnet_2" {
  vpc_id            = aws_vpc.my_vpc.id
  cidr_block        = var.subnet_2_cidr
  availability_zone = var.az_2

  tags = {
    Name = "main_subnet2"
  }
}        

• These resources define two subnets (subnet_1 and subnet_2) within the previously created VPC.
• Each subnet is associated with the VPC using the vpc_id attribute, ensuring they are part of the same networking environment.
• The cidr_block attribute specifies the CIDR block for each subnet, determining the range of IP addresses available for instances within that subnet.
• The availability_zone attribute specifies the availability zone where the subnet will be created. This allows distributing resources across multiple availability zones for high availability and fault tolerance.
• Tags are added to each subnet for identification purposes, making it easier to manage and identify resources within the AWS environment.

variable.tf

variable "vpc_cidr" {
  description = "10.0.0.0/16"
}

variable "subnet_1_cidr" {
  description = "10.0.1.0/24"
}

variable "subnet_2_cidr" {
  description = "10.0.2.0/24"
}

variable "az_1" {
  description = "us-east-1a"
}

variable "az_2" {
  description = "us-east-1b"
}        

• These variables are used to parameterize the Terraform configuration, allowing for flexibility and customization.
• vpc_cidr: Specifies the CIDR block for the VPC, defining the range of IP addresses for the VPC.
• subnet_1_cidr and subnet_2_cidr: Specify the CIDR blocks for the two subnets, defining the IP address ranges for instances within each subnet.
• az_1 and az_2: Define the availability zones where the subnets will be created. These variables specify the geographical locations within the AWS region where resources will be deployed, ensuring redundancy and fault tolerance.

As a conclusion,

Automating infrastructure deployment with Terraform and RDS snapshots offers numerous benefits, including increased efficiency and reproducibility. By manually creating a snapshot and then automating the deployment process, DevOps teams can seamlessly replicate environments while minimizing manual intervention and potential errors. This approach empowers teams to focus on innovation and delivering value to their organizations.