Troubleshooting _IPSEC VPN Lab on FortiGate NGFW(6.4) with Redundant ISP load Balancing-SD-WAN Interface

Before going into the Lab topology I would like brief about the IPsec VPN Tunnel formation and the type of messages exchanged in IKE Phase -1 and IKE Phase-2 . I have prepared the following diagrams which is specific to Lab topology .

• A VPN "tunnel" is the encrypted connection a VPN establishes so that traffic on the virtual network can be sent securely across the Internet. IPsec provides data integrity, basic authentication and encryption services to protect modification of data and unauthorized viewing by using Authentication Header (AH), Encapsulating Security Payload (ESP) and Internet Key Exchange (IKE) protocols. It operates in Transport and Tunnel Mode. In transport mode, the IP addresses in the outer header are used to determine the IPsec policy that will be applied to the packet. In tunnel mode, New IP header is added to provide extra layer of protection by defining Security policy to the inner IP packet.

Following diagrams are self explanatory regarding the IPsec process that happens in Phase-1 & Phase-2.Different fields in AH Header and ESP header are depicted. Important point to be noted here is SPI field which points to the respective Encryption and Authentication Algorithms.

Lab Topology: ( I have used GNS3,Fortigate 6.4 Image,Wireshark,CiscoIoS Router, Internet Cloud in this lab)

A user in the local NW of the Branch office (192.168.10.0/24) is trying to access the app_data of a server (192.168.12.0/24) in the HQ Data Centre site. Two sites are connected over an IPsec tunnel in the NW (192.168.99.0/24) with static routing. However, the user is not able to access the data as the IPsec tunnel is down due to multiple issues. 

Step-1 ( Verify L2/L3 Connectivity btw Peers):( Refer Pic_1) In the GUI of FortiGate NGFW I observed that IPsec VPN status is Inactive. We knew that IPsec is an L3 protocol it’s imp to have L2/L3 connectivity btw IPsec peers to establish the tunnel. So, in the very first step of troubleshooting, I sent a ping from Firewall in branch-office (99.2) to the IPsec tunnel endpoint (99.3) Firewall Int in HQ didn’t get any ICMP response. There was no echo reply so I have checked the Int status and observed that it is down. The interface is made up and crosschecked whether IPs are configured and reverified the static routes between two sites.

(Note: CLI Output images are attached)

Step-2:(Verify the Firewall Policies & NAT Mode to allow UDP traffic in both ends )

I sent a ping to the server in the HQ_LAN NW from the User in branch Ofc NW and observed that ICMP Packets are exchanged. However, the IPsec tunnel is not in Active state. IPsec uses UDP Port No-500(Without NAT) and 3500(With NAT) for establishing tunnel. So I checked the inbound and outbound policies observed that Implicit deny statement in both firewalls is dropping UDP traffic. So I have configured .So I have created Firewall policies at both ends to allow UDP and other traffic to form a IPsec Tunnel. Also, I made a NAT configuration error due to which NAT mode is unique and the tunnel is not establishing. So I reconfigured NAT so that NAT settings to be same on both ends and both uses UDP Port-500 in this lab. I rechecked the MTU size at both ends from logs and made sure it is same.

Created Policies to allow all traffic and Disabled NAT at both ends :

Finally, the IPsec Tunnel is active in both Firewalls(Sites).However, from the GUI mode I can see that data is not getting exchanged over IPsec Tunnel.

Step-3:( Phase-1 Troubleshooting, Pre-shared Key, Encryption, Auth Algorithm .IKE Version Mismatch ,Security Association Negotiation Failure )

In the IP Sec IKE Phase-1, we understood that Security Associations are exchanged and negotiated, and authenticated between IPsec Peers. So the Phase -1 IKE version, Pre-Shared Key, Authentication Algorithm, Encryption algorithm, Diffie Hellman group need to be configured as same in IPsec Peers. So I decided to verify these configurations in my topology. Instead of verifying the phase -1 settings in GUI I used CLI and debug commands/ messages to identify the problems. Note: Logs & reports feature in Fortinet GUI will give the debug msg report as well.

Debug Command -1 :" diagnose vpn tunnel list name <Phase-1 or phase2-name>" To view the phase-1 or 2status for a specific tunnel. I have used the above command in the the FortiGate CLI at Data Centre site.

Please refer the debug output screenshot that I have attached .

From the debug msg I have observed that Security Association bit "SA -0 " indicates there is mismatch between phase -1 selectors in IPsec peers or no traffic is being initiated. SA bit need to be 1 for successful SA establishment.

proxyid=To_Site_A proto=0 sa=0 ref=2 serial=3

Debug Command -2 : "diagnose vpn ike log filter name <phase1-name>"

Debug Command -3 : "diagnose debug app ike -1"

I have used the above command in the the FortiGate CLI at Data Centre site and from the debug output I have observed that there is a Preshared Key Mismatch from logs.

Due to mismatch in the preshared key IPsec peers are not able to authenticate each and other hence the security association is not negotiated . 

I have repeated the above debug commands in FortiGate Cli at Data Centre Site and in each iteration I have identified the error mgs - "Encryption, Auth Algorithm ,IKE Version Mismatch ,Security Association Negotiation Failure "from the debug output.

So I went back to the GUI mode of both Firewalls in two sites and made sure the Phase -1 Settings are same on both ends.

SA-0 in below Debug -O/P:

Preshared Key Mismatch Error in following debug O/P:

Debug O/P after resolving Pre-shared key mismatch :

Step-4:( Phase-2 Troubleshooting, Pre-shared Key, Encryption, Auth Algorithm ,Security Association Negotiation Failure :

We knew that In phase -2 IPsec tunnel Peers will perform a Diffie Hellman exchange a second time to generate a secret session key to send encrypted data. For this, the Encryption, Auth Algorithm, Key Life Time, Diffie Hellman group need to be the same in phase-2 settings in both FortiGate devices in two sites.

From debug commands, I have observed that there is a SA negotiation failure in phase-2 and I noticed that there is an encryption algorithm mismatch in Phase-2 settings.

Security Association Negotiation Failure:

After Troubleshooting the SA Negotiation Issue we get following debug o/p of successful SA Negotiation :

After performing all the above troubleshooting steps I have observed that the user can access the data from the server in the HQ_Data Centre . We can identify it in the IPsec VPN monitoring status of FortiGate Firewall upload and download status.

SD-WAN Feature in FortiGate Firewall ,Redundant ISP Connection on SD-WAN Interface to mitigate link failover and perform traffic load balancing on two ISPs. :

In the Data Centre Site I have configured the Port-4 & Port-5 as SD-WAN Interface and connected it to ISP-1 Gateway(192.168.0.1) and ISP-2 Gateway (172.16.0.1).Load Balancing algorithm - Source IP is set and I have configured the Google server in WAN status check to monitor the traffic load sharing.

With this configuration I was able to provide Redundant ISP connection to the Server hosted in the HQ_Data Centre to mitigate ISP link fail over and Load balancing.