'Q' stands for quick, not quirky! Connecting to your Azure instances securely with Megaport and Microsoft Azure ExpressRoute

Q-in-Q (sometimes known by its technical standards name; 802.1ad) traditionally finds its home in Service Provider Networks in order to provide greater flexibility in handling multiple VLANs destined for a specific target via a point to point Layer 2 circuit, or a VXC (Virtual Cross Connect) in Megaport language.

Stacking multiple VLAN tags in an Ethernet frame allows routing domain isolation (otherwise known as 'peering type') as the additional layer of tags identify and separate traffic. By using a different VLAN tag for each customer (outer tag) and subsequently peering type (inner tag), the traffic is identified within the frame and is transferred between A and B endpoints the service provider network, though treated as a single path for billing purposes.

Many (1, 2, 3, 4) words have been written on the above topic, mostly as a means to explain the service provider voodoo that is at play on networks where the carriage of multiple customer VLANs over a single VLAN is required. However, like all technology related items, the (implementation) devil is in the (implementation) detail and this topic had been one that had caused a significant amount of customer confusion, the majority of whom were not service providers themselves.

Customers who were not service providers themselves just wanted their connections to work from day one (and rightly so)!

Most customer networks, or at least those delivered across Megaport, are build with standard VLAN tags (802.1q, or 'dot1q') encapsulations providing two main factors for service differentiation; namely 'where is the B-end of this service going to' and 'what is the rate limit/policer to be applied to this circuit'. So if you have a physical access port (Network/Network interface, or NNI) at a given location, you may apply one or more VLANs that define where the B-end of the service will terminate, whether that's your own port in another facility (across the metro, state or even internationally), this is known as a 'Private VXC', or your targets are already established ports of the Cloud Service providers ('Cloud' target) or other third parties who make their ports available across the Megaport ecosystem ('Marketplace' and reverse-billed 'Service Key' definitions below). Or if you haven't heard of the Internet Exchange (known as 'MegaIX' in most markets) - get onto it!

With me still? Great! You're probably either a customer already, or shortly about to become one. Let's talk about the 'quirky' bit that comes into the above however.

The above definition of a VXC/dot1q VLAN providing two factors (A/B end pairing of a point to point link and size dimensioning) is still true, but these two factors combined also provide the key to how these circuits are billed. On the Megaport side of the equation, there is a third factor that combines with the above two that is the 'time at x rate, for a given A/B pair' that we use to determine the charging - and yes you may vary this up or down in the majority of cases, to suit your specific bandwidth requirements. Take a look at our explainer on how this is calculated, check the interactive pricing tool or try it out by ordering a service in the portal. I didn't profess to make this post about pure dollars alone!

If you're a service provider, or a customer that just wishes to extract maximum value from your shiny new VXC, you may think "instead of me paying a per VLAN charge and then a bandwidth charge, how about I roll up multiple VLANs into a single VXC" and this is in fact absolutely a great way to use the Megaport service offerings. There's a few reasons this is attractive, firstly our generous 9100 byte MTU means the extra overheads presented by Q-in-Q isn't generally going to be an issue that will impact customer traffic (fragmentation is bad). Secondly a VXC is by definition path-protected and therefore even if a fibre path that we use is impacted (backhoes are known to be fibre seeking, despite the best Dial-before-you-dig efforts) generally our network will intelligently route around this issue and failover should be hitless, or at most a few milliseconds of extra latency. Thirdly (and lastly) if you want even better resilience (and you should absolutely be thinking about this if your production networks are relying on these paths being available) there are the 'within site' resilience considerations of LACP groups/LAG bundles (Nx10G physical ports of between 2 and 8 per group) and of course our new port diversity zone functionality - or better yet, why not both, for that traffic that 'absolutely positively has to stay live under all circumstances'.

Let's take another quick look at that technical specification tear sheet for the VXC (above) and dive into what is needed to make this happen. It calls out the 9100 byte MTU covered in the previous paragraph, but all that is needed is to present to Megaport a Q-in-Q/802.1ad frame with an outer Ethertype/TPID value of 0x8100 (standard dot1q) - you of course may configure the inner tags (so called customer, or C-tags) as 0x8100, 0x88A8 OR 0x9100 - the value is passed transparently from end to end based on the service provider, or S-tag, which is the outer value in this case. Pretty amazingly easy to get going, and if bandwidth isolation isn't required between VLANs, then this is an ideal scenario for trunking multiple VLANs between two locations that are Megaport connected.

Of course, if handling multiple tags isn't your thing, and you just want to treat the physical port like it were a fibre connection (still path diverse however) you can toggle the 'untag' option in the portal when creating a VXC and instantly any VLANs you send our way will traverse its way across our network untouched, and bonus points because using this feature means that the outer tag doesn't even need to match 0x8100 for it to be carried (though the side-note here is that the entire physical interface does get committed to a single B-end target).

One particular use case of the above (Q-in-Q, 0x8100 outer, 0x8100 inner) is that of Microsoft ExpressRoute. A VXC connecting to Microsoft ExpressRoute target may contain one or two inner VLAN tags. These are the C-tags configured in the Microsoft Azure console for the specified peering type/s. Note that Q-in-Q is a requirement in this scenario, even if only a single Microsoft edge device (primary or secondary) or peering type (private or Microsoft) is employed.

Can your network device handle Q-in-Q? The answer is (it may surprise you) more than likely yes, a quick visit to your friendly vendor's specification sheet will usually answer this. However with the devil lying most definitively in the detail and whilst the majority of spec sheets will list a very happy looking checkmark alongside Q-in-Q/802.1ad. In practice, due to the nature of combining dot1q/single tag VLANs and 802.1ad/Q-in-Q VLANs on the same physical interface (so called 'selective Q-in-Q', not so well published in most vendor spec sheets) customers have ended up using one of the workarounds above (up to and including tag management via 'untag' option) as well as the lesser preferred 'have you tried plugging a loopback cable in on the switch' to allow different tag depth schemas to apply on a single device. If the concept alone scares you, please look away for a second:

But never fear, help is now at hand, thanks to a new feature in configuring VXCs to Azure, known as 'Q-in-Q breakout'.

For Microsoft Azure ExpressRoute connections, the Megaport optimised experience in presenting a Microsoft Service Key (and associated primary and secondary target endpoint selection) provides an ability to now breakout the encapsulated C-Tag assigned for a specified Azure peering type. This functionality is enabled by toggling on the Configure single Azure peering VLAN in the Connection Details panel of a new ExpressRoute connection, after selecting a primary or secondary target B-End port. The associated VLAN will then be landed upon the customer port as a standard 802.1q/dot1q VXC.

By enabling the Azure VLAN breakout functionality, you are requested to input an Azure peering VLAN that will match with the value that you configure (within the Azure portal) upon the selected Peering type configuration from the Azure ExpressRoute configuration. This can be achieved via the Microsoft Azure portal or other configuration method such as PowerShell. For details, you can refer to the Tutorial: Create and modify peering for an ExpressRoute circuit using the Azure portal.

If you leave the Preferred A-End VLAN field blank, a randomly chosen VLAN will be assigned for your VXC when you place the order, otherwise enter a VLAN and a validation check will be performed to ensure this VLAN is available.

Great stuff! Now which option is best for me?

The below flowchart shows a decision path to assist in selecting the best Q-in-Q method for your environment. It lists options 1 through 5 in order of preference. The Microsoft Azure ExpressRoute SLA requires a redundant Layer 3 connectivity configuration. Options 1 through 4 provide this redundancy, and option 5 can provide it if you use two physical port demarcations.

You can read more on the new feature and how it can suit your requirements at the following pages. Thanks to all of my Megaport colleagues that made this possible for our customers!

• Configuring Q-in-Q https://docs.megaport.com/connections/q-in-q/
• ?Connecting to Microsoft Azure ExpressRoute https://docs.megaport.com/cloud/megaport/microsoft/