Transit Gateway Setup and VPN to Datacenter

Using a predefined CloudFormation template, we will deploy a Cisco Router into the Simulated Datacenter VPC, as well as the Transit Gateway.

Deploy Transit Gateway and Datacenter Router

Run CloudFormation template 2.tgw-csr.yaml to deploy the Transit Gateway, route Tables, and the Datacenter Router

1. CloudFormation Launch link below that corresponds to the AWS Region in which you deploy the first stack.

The Specify stack details give the stack a name (compounded names work well. i.e. if the VPC stack created in the setup module was named TGW1 name this stack TGW1-CSR), pick the keypair you created earlier, and enter the name of your first stack (must be entered exactly to work). Click Next.

Configuration stack options we don’t need to change anything, so just click Next in the bottom right.

Scroll down to the bottom of the Review name_of_your_stack and check the I acknowledge the AWS CloudFormation might create IAM resources with custom names. Click the Create button in the lower right.

Wait for the Stack to show Create_Complete.

Access CSR using AWS Cloud9 Environment

In order to access and work on the Datacenter router, we will use AWS Cloud9. Cloud 9 is an IDE platform that is managed as a service for us. We will be using just a fraction of its capability. But this intro should give you some ideas how you can provide a shared environment for managing your Infrastructure as code development.

1. In the AWS Management Console change to the region you are working in. This is in the upper right hand drop down menu.
2. In the AWS Management Console choose Services then select Cloud9.

From Your Environments page click the Open IDE button on the Workshop Environment Box.

1. This will bring up the Cloud9 Console and download the github repo to your working folder.
2. From the file menu select Upload Local Files… and click Select files button, navigate to the key file you created earlier. note: it should have a .pem extension. In the Cloud9 console, the file will show up in the left-hand folder list.

1. In the main panel click the x sign next to the welcome tab to close this tab.
2. In the main panel click the + sign and launch a New Terminal. This is a bash shell on the Cloud9 Instance.
3. move the key to the .ssh folder: mv _key_name_.pem ~/.ssh/
4. restrict access to the key file: chmod 400 ~/.ssh/_key_name_.pem

From another browser tab, again navigate to the Management Console and choose Services then select CloudFormation.

From the left-hand menu, select Exports in the left hand menu and find the export for ssh to the CSR: DC1-stack-name-CSRssh and copy the Export value

Back on the Cloud9 Browser tab paste this into the bash shell. note: in the command you will notice the -i reference to the pem file you just copied, this is the private half of the key pair. The public key is on the Cisco CSR. Answer yes to Are you sure you want to continue connecting (yes/no)?

You now are connected to the Cisco CSR in the Datacenter VPC. We will be configuring the Cisco CSR.

Lets look at the Interfaces by typing a the #prompt: show ip interface brief or sh ip int br for short. You will see the GigabitEhternet1 which is the interface our ipsec tunnel were traverse.

Take a look at the route table on the CSR by typing at the #prompt: sh ip route. You will see S* 0.0.0.0/0 which is a static default route pointing to the 10.4.0.1 address. This is the local VPC router which will connect the Interface to the Internet Gateway and use an Elastic IP address (The Elastic IP will be used as the Customer Gateway IP address when we setup the VPN between the datacenter and the Transit Gateway). The public Elastic IP is a one-to-one mapping to the private 10.4.x.x IP address you just SSHed to.

Setup VPN Between Datacenter and Transit Gateway

Ipsec tunnels can be setup over the internet or over Direct Connect (using a Public Virtual Interface). In this case we are connecting over the public backbone of AWS. We will create two VPN tunnels from the Transit Gateway and connect them into a single instance of the Cisco CSR in the Datacenter. In a real production environment we would setup a second router for redundancy and for added bandwidth setup multiple tunnels from each Cisco CSR (or whichever ipsec device you use). Each ipsec tunnel provides up to 1.25Gbps. This is called Equal cost multipath routing. On the AWS side, up to 50 parallel (ECMP) paths are supported. Many vendors support 4–8 ECMP paths, so check with your vendor)

In the AWS Management Console change to the region you are working in. This is in the upper right hand drop down menu.

In the AWS Management Console choose Services then select VPC.

From the menu on the left, Scroll down and select Transit Gateway Attachments.

You will see the VPC Attachments listed, but we want to add one to connect our Datacenter. Click the Create Transit Gateway Attachment button above the list.

Fill out the Create Transit Gateway Attachment form. exactly as below (note: these choices will match our config of the router on the other side of the VPN tunnels)

• Transit Gateway ID will have a name Tag matching your first CloudFormation Stack name.
• Attachment Type is VPN
• Customer Gateway (CGW) will be Existing. note: the CloudFormation template created the CGW. it is the IP address of our Datacenter VPN device and in the lab matches the EIP of EC2 Instance running the Cisco CSR.
• Leave Routing options set to Dynamic(requires BGP). note: BGP is required if you want traffic to balance across more than one VPN tunnel at a time (ECMP or Equal Cost Multipathing)
• For Inside IP CIDR for Tunnel 1 use 169.254.10.0/30 for CIDR.
• For Pre-Shared Key for Tunnel 1 use awsamazon
• For Inside IP CIDR for Tunnel 2 use 169.254.11.0/30for CIDR.
• For Pre-Shared Key for Tunnel 2 use awsamazon
• Once the page is filled out, click Create attachment at the bottom right.

While we are on the Transit Gateway Attachments page, lets go back to the top and give the VPN connection a name. Scan down the Resource type column for the VPN Attachment. *note: you may have to hit the refresh icon in the upper right above the table to get the new VPN to show. If you click the pencil that appears when you mouse over the Name column, you can enter a name. Be sure to click the check mark to save the name.

From the Menu on the Left Select Site-to-Site VPN Connections. From the main panel, you likely will see the VPN is in State pending. That fine. Lets take a look toward the bottom, and click the Tunnel Details tab. Record the two Outside IP Addresses. We want to record them in the order of the one pairing up with the Inside IP CIDR range 169.254.10.0/30 first. note: You can use cloud9 as a scratch pad, by clicking the + in the main panel and selecting New file. be sure to paste them in the right order!

From the menu on the left, Scroll down and select Transit Gateway Attachments. We need to verify that the attachment we created above is no longer in status pending. Instead it should be is state available like all of the VPC attachments in the list.

From the Menu on the Left Select Transit Gateway Route Tables. From the table in the main panel select Green Route Table. Lets take a look toward the bottom, and click the Associations tab. Associations mean that traffic coming from the outside toward the Transit gateway will use this route table to know where the packet will go after routing through the TGW. note: An attachment can only be Associated with one route table. But a route table can have multiple associations. Here in the Green Route Table, We already have one association, The Datacenter Services VPC. Click Create associations in the Associations tab. From the drop-down list, select the vpn. note:it should be the only one in the list without a Association route table . Click Create association.

While at the Transit Gateway Route Tables, take a look at the Propagations tab. These are the Resources that Dynamically inform the route table. An attachment can propagate to multiple route tables. For the Datacenter, we want to propagate to all of the route tables so the VPC associated with each route table can route back to the datacenter. Lets start with the Green Route Table. We can see all of the VPCs are propagating their CIDR to the route table. Since the Datacenter Services VPC is also associated with this route table, we need to propagate the VPN routes to the Green Route Table.

Click in Create Propagation on the field “chose attachment to propagate”, select the attachment of the VPN (previously named by you) and click in Create propagation.

Repeat the above step on the propagations tab for the Red Route Table and the Blue Route Table.

Take a look at each of the route tables and notice the tab Routes. You can see the routes that are propagated, as well as a static route table that was created for you by the CloudFormation template. That’s the default route (0.0.0.0/0) that will direct traffic destined for the internet to the Datacenter Services VPC and ultimately through the NAT Gateway in that VPC. note: there is also a route table with no name. This is the default route table. In this lab we do not intend to use the default route table.

Back on the Cloud9 browser tab, using the two VPN tunnel endpoint address generated from the step above, cd to tgwwalk on the Cloud9 bash console and run the bash script, ./createcsr.sh. note: Be sure to put the address that lines up with Inside IP CIDR address 169.254.10.0/30 for ip1. Example from Site-to-Site VPN

note: AWS generates starter templates to assist with the configuration for the on-prem router. For your real world deployments, you can get a starter template from the console for various devices (Cisco, Juniper, Palo Alto, F5, Checkpoint, etc). Word of Caution is to look closely at the routing policy in the BGP section. you may not want to send a default route out. You likely also want to consider using a route filter to prevent certain routes from being propagated to you.

1. On the left hand panel, the output file should be listed. You may have to open the tgwwalk folder to see the txt file. Select all text (ctrl-a on pc/command-a on mac). Then copy the text to buffer (Select andy copy all text (ctrl-a and then ctrl-c on pc/command-a and then command-c on mac))
2. using a bash tab in cloud9, ssh back into the CSR. note: the ssh command for the CSR is given for you from the Exports menu in CloudFormation.
3. enter configuration mode by entering config t, which will take you to a config prompt

1. Once in Configuration mode note: you should see (config)# prompt, paste all text (ctrl-v on pc/command-v on mac) from the outputfile created in step 4. This will slowly paste into the configuration.
2. Once, the paste is finished, if you are still at the (config)# or (config-router) prompt, type end and press enter.
3. Now lets look at the new interfaces: sh ip int br. You should see new interfaces: Tunnel1 and Tunnel2 and they both should show up. *note: if they do not change from down to up after a minute, likely cause is the ip addresses were flipped in the createcsr script.

Lets make sure we are seeing the routes on the Cisco CSR. first we can look at what BGP is seeing: show ip bgp summary. The most important thing to see is the State/PfxRcd (Prefixes received). If this is in Active or Idle (likely if neighbor statement is wrong: IP address, AS number) there is a configuration issue. What we want to see is a number. In fact if everything is setup correctly we should see 4 for each neighbor.

We can also see what those routes are and how many paths we have with the show ip routes or sh ip ro command.

Notice that there is only one next-hop address for each of the VPCs CIDRs. We can fix this by allow Equal Cost Multipathing (ECMP). Back in config mode (config t) we will set maximum-paths to 8 in our BGP router:

Now, run sh ip ro command again. See, both the tunnels are showing up!

Just to verify where those routes are coming from, we can take a look at the Green Route Table. note: remember, it’s under the VPC service and Transit Gateway Route Tables at the bottom of the left menu. There should be 5 routes listed. Any ideas why only 4 show up on the CSR?

Routing in the VPCs

Create Routes in the VPC to the Transit Gateway Attachments

The CloudFormation Template created attachments to the VPCs and route tables for the transit gateway. We need to setup routing within the VPC. What traffic do we want going from each subnet to the Transit Gateway.

1. In the AWS Management Console change to the region you are working in. This is in the upper right hand drop down menu.
2. In the AWS Management Console choose Services then select VPC.
3. From the menu on the left, Scroll down and select Route Tables.
4. You will see the Route Tables listed in the main pane. Lets Start with NP1-_stackname-Private route table, Check the box next to it. Let take a look toward the bottom of the panel and click the Routes tab. Currently, there is just one route, the local VPC route. Since the only way out is going to be the Transit Gateway, lets make our life simple and point a default route to the Transit Gateway Attachment. Click the Edit Routes in the Routes tab.
5. On the Edit routes page, Click the Add route button and enter a default route by setting the destination of 0.0.0.0/0. In the Target drop-down, select Transit Gateway and pick your Transit Gateway create for this project. It should be the only one. _Note: The route tables NP2-_stackname-Private and P1-_stackname-Private already have this default route create as part of the setup, so you use them for comparison.

1. For the DCS1-_stackname-Public and DCS1-_stackname-Private where our NAT Gateway is, we need a special route. We already have a default route pointed at the Internet Gateway(IGW) for the public and to the Nat Gateway(NGW) for the private to get to the internet, so we need a more specific entry to route internally. Lets use the rfc 1918 10.0.0.0/8 CIDR as that can only be internal and allows for future expansion without changes. Follow the steps above to create the route for destination 10.0.0.0/8 in the DCS1-_stackname-Private route table. _Note: DCS1-_stackname-Public already have this route create as part of the setup, so you use it for comparison._ Be sure not to alter the 0.0.0.0/0 route pointed to the IGW org NGW for these route tables.
2. Because the CloudFormation template setup a Security Group to allow ICMP traffic from 10.0.0.0/8, we should now be able to test pings from lots of place.
3. In the AWS Management Console choose Services then select EC2.
4. From the menu on the left, Scroll down and select Instances.
5. In the main pane, select the NP1 EC2 instance from the list and copy its ip address down. You can repeat for other servers (NP2, P1, and DC1) if you want to test connectivity to other VPCs as well. First, we want to test connectivity to the P1 server. Remember, we do not want our non prod instances to be able to reach our production servers or vice-versa.

• You can also get a list of IPs using the AWS CLI which you can run from the Cloud9 Instance. Here’s one that extracts out all of them. The 10.16.x.x is the NP1, the 10.8.x.x is the P1 instance.

1. In the AWS Management Console choose Services then select Systems Manager. Systems Manager Gain Operational Insight and Take Action on AWS Resources. We are going to take a look a just one of seven capabilities of Systems Manager.
2. From the menu on the left, Scroll down and select Session Manager. Session Manager allows us to use IAM role and policies to determine who has console access without having to manage ssh keys for our instances.
3. In the main pane, click the Start session button. Pick an Instance to shell into. You will now enter a bash shell prompt for that instance.
4. Let Ping a server. Every one second or so, you should see a new line showing the reply and roundtrip time.

verify Internet access by using the curl command on the NP1, NP2 or P1 (the Datacenter Server wont use the Transit Gateway to get to the internet, but should still work). If you curl https://cloudformation.us-east-1.amazonaws.com it should return healthy.