Creating AWS VPC Endpoints with Terraform: A Step-by-Step Guide

Here's How My File Structure looks

terraform-project/
├── main.tf
├── variables.tf
├── terraform.tfvars
├── networking/
│   ├── main.tf
│   ├── variables.tf
├── ec2/
│   ├── main.tf
│   ├── variables.tf
├── vpc-endpoint/
│   ├── main.tf
│   ├── variables.tf        

Step 1: Setting Up the VPC

First, create the main configuration file for your Terraform project and define the VPC module.

main.tf

module "networking" {
  source   = "./networking"
  vpc_cidr = var.vpc_cidr
  vpc_name = var.vpc_name
}        

networking/main.tf

# Setup VPC
resource "aws_vpc" "dev_proj_1_eu_central_1" {
  cidr_block = var.vpc_cidr
  tags = {
    Name = var.vpc_name
  }
}        

variables.tf

variable "vpc_name" {
  type        = string
  description = "DevOps Project 1 VPC 1"
}

variable "vpc_cidr" {
  type        = string
  description = "VPC CIDR block"
}        

terraform.tfvars

vpc_cidr = "11.0.0.0/16"
vpc_name = "dev-proj-jenkins-eu-west-vpc-1"        

Step 2: Creating Subnets

Next, create public and private subnets within the VPC.

main.tf

module "networking" {
  source               = "./networking"
  vpc_cidr             = var.vpc_cidr
  vpc_name             = var.vpc_name
  cidr_public_subnet   = var.cidr_public_subnet
  eu_availability_zone = var.eu_availability_zone
  cidr_private_subnet  = var.cidr_private_subnet
}        

networking/main.tf

# Setup public subnet
resource "aws_subnet" "dev_proj_1_public_subnets" {
  count             = length(var.cidr_public_subnet)
  vpc_id            = aws_vpc.dev_proj_1_vpc_eu_central_1.id
  cidr_block        = element(var.cidr_public_subnet, count.index)
  availability_zone = element(var.eu_availability_zone, count.index)
  tags = {
    Name = "dev-proj-public-subnet-${count.index + 1}"
  }
}

# Setup private subnet
resource "aws_subnet" "dev_proj_1_private_subnets" {
  count             = length(var.cidr_private_subnet)
  vpc_id            = aws_vpc.dev_proj_1_vpc_eu_central_1.id
  cidr_block        = element(var.cidr_private_subnet, count.index)
  availability_zone = element(var.eu_availability_zone, count.index)
  tags = {
    Name = "dev-proj-private-subnet-${count.index + 1}"
  }
}        

variables.tf

variable "cidr_public_subnet" {
  type        = list(string)
  description = "Public Subnet CIDR values"
}

variable "cidr_private_subnet" {
  type        = list(string)
  description = "Private Subnet CIDR values"
}

variable "eu_availability_zone" {
  type        = list(string)
  description = "Availability Zones"
}        

terraform.tfvars

cidr_public_subnet   = ["11.0.1.0/24", "11.0.2.0/24"]
cidr_private_subnet  = ["11.0.3.0/24", "11.0.4.0/24"]
eu_availability_zone = ["eu-west-1a", "eu-west-1b"]        

Step 3: Setting Up the Internet Gateway

Create an Internet Gateway and attach it to the VPC.

networking/main.tf

# Setup Internet Gateway
resource "aws_internet_gateway" "dev_proj_1_public_internet_gateway" {
  vpc_id = aws_vpc.dev_proj_1_vpc_eu_central_1.id
  tags = {
    Name = "dev-proj-1-igw"
  }
}        

Step 4: Configuring Route Tables

Create and configure route tables for the public and private subnets.

networking/main.tf

# Public Route Table
resource "aws_route_table" "dev_proj_1_public_route_table" {
  vpc_id = aws_vpc.dev_proj_1_vpc_eu_central_1.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.dev_proj_1_public_internet_gateway.id
  }
  tags = {
    Name = "dev-proj-1-public-rt"
  }
}

# Private Route Table
resource "aws_route_table" "dev_proj_1_private_route_table" {
  vpc_id = aws_vpc.dev_proj_1_vpc_eu_central_1.id
  tags = {
    Name = "dev-proj-1-private-rt"
  }
}        

Step 5: Associating Subnets with Route Tables

Associate the public and private subnets with their respective route tables.

networking/main.tf

# Public Route Table and Public Subnet Association
resource "aws_route_table_association" "dev_proj_1_public_rt_subnet_association" {
  count          = length(aws_subnet.dev_proj_1_public_subnets)
  subnet_id      = aws_subnet.dev_proj_1_public_subnets[count.index].id
  route_table_id = aws_route_table.dev_proj_1_public_route_table.id
}

# Private Route Table and Private Subnet Association
resource "aws_route_table_association" "dev_proj_1_private_rt_subnet_association" {
  count          = length(aws_subnet.dev_proj_1_private_subnets)
  subnet_id      = aws_subnet.dev_proj_1_private_subnets[count.index].id
  route_table_id = aws_route_table.dev_proj_1_private_route_table.id
}        

Step 6: Setting Up EC2 Instances

Create and configure EC2 instances within the public and private subnets.

main.tf

module "ec2-public-subnet" {
  source                   = "./ec2"
  ami_id                   = var.ec2_ami_id
  instance_type            = "t2.medium"
  tag_name                 = "EC2 Instance: Public Subnet"
  public_key               = var.public_key
  subnet_id                = tolist(module.networking.dev_proj_1_public_subnets_ids)[0]
  sg_for_jenkins           = [module.security_group.sg_ec2_sg_ssh_http_id, module.security_group.sg_ec2_jenkins_port_8080]
  enable_public_ip_address = true
  key_name                 = "aws_ec2_terraform_public"
}

module "ec2-private-subnet" {
  source                   = "./ec2"
  ami_id                   = var.ec2_ami_id
  instance_type            = "t2.medium"
  tag_name                 = "EC2 Instance: Private Subnet"
  public_key               = var.public_key
  subnet_id                = tolist(module.networking.dev_proj_1_private_subnets_ids)[0]
  sg_for_jenkins           = [module.security_group.sg_ec2_sg_ssh_http_id, module.security_group.sg_ec2_jenkins_port_8080]
  enable_public_ip_address = false
  key_name                 = "aws_ec2_terraform_private"
}        

ec2/main.tf

variable "ami_id" {}
variable "instance_type" {}
variable "tag_name" {}
variable "public_key" {}
variable "subnet_id" {}
variable "sg_for_jenkins" {}
variable "enable_public_ip_address" {}
variable "key_name" {}

resource "aws_instance" "jenkins_ec2_instance_ip" {
  ami           = var.ami_id
  instance_type = var.instance_type
  tags = {
    Name = var.tag_name
  }
  key_name                    = var.key_name
  subnet_id                   = var.subnet_id
  vpc_security_group_ids      = var.sg_for_jenkins
  associate_public_ip_address = var.enable_public_ip_address

  metadata_options {
    http_endpoint = "enabled"
    http_tokens   = "required"
  }
}        

Step 6: Create S3 in VPC

Accessing an S3 bucket from a private EC2 instance

Step 7: Creating VPC Endpoints

Set up a VPC endpoint for S3 to enable private EC2 instances to access the S3 service.

If you're using a service other than Amazon S3 or DynamoDB, ensure you're using the correct endpoint type, as gateway endpoints are specifically designed for these services. For other services, use interface endpoints, which provide private connectivity and are supported by a wide range of AWS services, including EC2, ECS, and Secrets Manager. Note that interface endpoints create an elastic network interface (ENI) in your subnet, allowing for private connectivity using private IP addresses.

main.tf

module "vpce_endpoint" {
  source            = "./vpc-endpoint"
  vpc_id            = module.networking.dev_proj_1_vpc_id
  service_name      = "com.amazonaws.eu-west-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [module.networking.dev_proj_1_private_route_table_ids]
}        

vpc-endpoint/main.tf

variable "vpc_id" {}
variable "service_name" {}
variable "vpc_endpoint_type" {}
variable "route_table_ids" {}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = var.vpc_id
  service_name      = var.service_name
  vpc_endpoint_type = var.vpc_endpoint_type
  route_table_ids   = var.route_table_ids
  tags = {
    Name = "dev-proj-1-vpce-s3"
  }
}        

variables.tf

variable "vpc_id" {
  type        = string
  description = "VPC ID"
}

variable "service_name" {
  type        = string
  description = "AWS Service Name for VPC Endpoint"
}

variable "vpc_endpoint_type" {
  type        = string
  description = "VPC Endpoint Type (Gateway/Interface)"
}

variable "route_table_ids" {
  type        = list(string)
  description = "List of Route Table IDs"
}        

After establishing a secure connection to the VPC endpoint, we can now access and list the S3 buckets.

Conclusion

This guide walks you through setting up a VPC, creating subnets, configuring route tables, and finally creating VPC endpoints using Terraform. By following these steps, you should have a functional VPC setup with public and private subnets, route tables, and VPC endpoints, enabling secure and efficient communication between your AWS resources.

If you need further customization or run into issues, feel free to ask for more specific guidance!