It's Always the Network

If you have ever worked on a computer network, whether you are a Network Administrator, Network Engineer, Help Desk, or pretty much any other IT related position, you have heard that something is always going on with the Network. In the era of the cloud, networking is becoming even more critical, with more and more enterprises shifting business critical applications or traditional services like; files, Active Directory, DNS, etc. to the cloud. How can one ensure that your network is not actually the problem?

Can I Troubleshoot Networking in the cloud?

Your infrastructure is running in the cloud and all is well, or at least that is what you think until you start getting a flood of emails from co workers throughout your company; "I cannot seem to access the files located on X, do you know why?", "Application B keeps telling me that it cannot read database Y, what's going on with the network?", and so on....... Question is...how would you troubleshoot something like this in the cloud? The answer is not always as easy as you'd expect when it comes to the more traditional model of datacenter or even infrastructure monitoring. Most likely, in the realm of datacenter networking or physical networking, you likely have an array of tools at your disposal, such as; PRTG, Solarwinds, NetScout, NetFlow, ManageEngine, Nagios, Zabbix, etc. However, cloud providers abstract away the physical networking infrastructure, so you do not have any OSI Layer 2 to help gain insights into what is going on within your cloud network.

Where do I start?

Even though you do hand some amount of control over the the CSP (Cloud Service Provider), they do not leave you empty handed. The cloud providers have native tooling that helps provide insights into how your cloud networking is performing, capturing many of the metrics, insights and data flows that you are used to. Additionally, new network monitoring tools have come onto the market that boast a fairly heavy ability to monitor cloud networking. Companies like Cisco, ThousandEyes and others utilize the same data you would get from the cloud native, but add their own GUI on top and add some vendor specific features. From here on, I will be referencing Network monitoring from within the Microsoft Azure platform specifically. AWS and GCP do have their own native tools and I encourage you to explore their documentation to gain insights into their specific monitoring practices.

Azure Monitor for Networks (preview)

All things monitoring within Azure start with the Azure Monitor service, however this is not where Network monitoring in Azure began oddly enough. Suffice it to say though, they have at least added some network health monitoring within Azure Monitor. This new feature, which is in preview, gives you visibility into all your network resources within Azure. Do you want to know the health of your ExpressRoute circuits? Is a VPN Tunnel connection unavailable? You can quickly search or filter on Subscription, ResourceGroup or ResourceType to see the health of your various networking resources.

The other cool feature within Azure Monitor for Networks is the Dependency view (currently supported by AppGateway, Azure Virtual WAN and Load Balancer). This feature gives you insight into how the resource is configured and view the various listeners, rules and front end IP addresses are connected, ensuring the right traffic is getting to the right back end resources. If your back end resources happen to be Virtual Machines/Virtual Machine Scale Sets, then you have some nice right click options to further troubleshoot any potential issues with VM insights or Network Watcher connections.

Network Performance Monitor

What may feel like ages ago, but was really only about 3-4 years ago, Microsoft announced two monitoring solutions for Azure Networking, NPM and Network Watcher. NPM was billed as a way to monitor hybrid network connections in Azure, on premises locations, to Microsoft Cloud services and ExpressRoute connections. NPM is a very powerful monitoring tool, with the ability to monitor mission critical multi tier applications, multiple datacenters, branch offices, etc. The NPM solution uses the Log Analytics agent to be installed on at least one sub network that has a Windows Server or Windows Desktop client (*Yes, NPM can monitor via Linux based machines, however that is only for the Performance Monitoring feature, it does not support Service Monitoring or ExpressRoute).

After you install the agents where you want to configure them for monitoring, you can choose either TCP or ICMP, personally I would choose TCP, as typically ICMP traffic is denied by corporate firewalls. After you configure your Log Analytics agents, you then would go to the Azure Portal and search for the NPM solution within the Azure marketplace or you can add it via the Azure Monitor Solution Gallery:

Or you can install it via Azure CLI/Cloud Shell:

az monitor log-analytics solution create --name
                                         --plan-product
                                         --plan-publisher
                                         --resource-group
                                         --workspace
                                         [--no-wait]

                                         [--tags]

Once you have installed the NPM solution, you will configure the solution to begin monitoring. There is much more detail to go through to setup NPM specifically to your environment, so I encourage you to check out:

Azure Network Watcher

To add on top of the capabilities of NPM, Microsoft also announced Network Watcher as a cloud first network monitoring solution. Network Watcher monitors, diagnoses, views metrics and enables/disables logs for resources within an Azure vNet. Now this strictly means that Network watcher can only view Infrastructure as a Service workloads, it is not intended nor will it work for PaaS services. Despite this limitation, the capabilities for Azure Network Watcher allow you to view your IaaS topology,

troubleshoot connections with Connection Monitor; setting tests from an Azure virtual machine to either another Virtual machine or a location you specific manually via URI, FQDN or IP address. This feature allows you to also specify the source and destination ports to ensure you are verifying connectivity of a service or application that uses that specific TCP port.

Enable packet capturing on Azure Virtual NIC's to troubleshoot application connectivity issues.

With IP flow verify, you can ensure a packet is allowed or denied through your Network Security Groups. The flow verify ensures that the traffic flow gets by the NSG, checking all of the values within the 5-tuple.

Check the effective Network Security Group rules, ensuring the correct rules are associated with the Virtual Machine NIC.

Troubleshoot VPN connections with automated VPN checks on your Virtual Network Gateways and Connection resources.

Verify your next hop routing by ensuring the traffic follows the appropriate routing within Azure. Whether you specify a Network Virtual Appliance, Virtual Network, Internet or Virtual Network Gateway, this checks where the Virtual Machine is sending its traffic.

Phew, that's quite a lot and there is still more. The biggest feature in my humble opinion is the NSG Flow logs, i.e. Netflow in the cloud. If you have deployed Network Security Groups within your virtual networks, you can monitor the 5-tuple network flow to see what protocols are traversing your cloud network and where that traffic is coming from.

For any Azure IaaS workload deployments, having Network Watcher enabled is a must. It ensures that the Virtual Machines have the Network Watcher extension installed, which will only help enhance the capabilities of troubleshooting your VM routing issues.

Azure Networking Monitoring 2.0?

Wait.... what? Well, given the nature of Microsoft Azures hybrid first approach, Microsoft is making several enhancements to Network Watcher. First among those is to bring in NPM under Network Watcher, for a single pane of glass network monitoring. Removal of this additional interface and centralizing that monitoring eases the burden of monitoring.

With the addition of the NPM tool set to Network Watcher, new capabilities are now being developed within the connection monitor. You are now able to facilitate connectivity testing in Azure and hybrid network connections, testing connectivity over HTTP, HTTPS or ICMP to a variety of endpoints.

Summary

The cloud provides many different ways for Networking professionals to monitor, troubleshoot and act on the network resources that have been configured. While the traditional SNMP, OSI layer 2 has been abstracted away from you, there is still a rich monitoring tool set that is now available that had not quite been there before, giving you greater insight into how your entire network is performing. I certainly will not say you need to use the cloud native tools. If you have an existing tool set already and it has the capabilities to monitor you network resources in the cloud, then obviously you should consider that as option 1. That being said, I wouldn't dismiss the many native cloud network monitoring tools. Given some of the focus to extend the capabilities to monitor some on premises/datacenter networks, they are worth consideration to add to any IT Network's professional tool bag.