#Avi-First-Experience
I am writing this blog to share my perspective and first impressions about NSX Advanced Load Balancer, the erstwhile Avi Platform acquired by VMware. I got the opportunity to work on the Avi Application Delivery Controller (ADC) platform when I joined Avi Networks few years back. My admiration has continuously grown over these years looking at how the platform has evolved over time, leaving all the traditional players and market leaders way behind. I could safely say that the startup culture and the momentum in innovating is very well nurtured with the acquisition by VMware. The NSX Adv LB (Avi) solution is a mature enterprise class ADC platform which is capable of supporting almost all use cases for modern application delivery.
Simplicity is the ultimate sophistication
- Leonardo DaVinci
The very words that define the NSX Advanced Load Balancing (Avi) solution. The UI screenshots provided in the article underscores this fact. Click through the hyperlinks if you wish to learn more about the topic.
The ADC marketplace is crowded with so many players and it has become a challenging task to invest on the right solution that could meet all the enterprise use cases as well as the future requirements of scale. I would like to summarise my criteria for technical evaluation for choosing the right ADC platform in the table given below -
Though not a comprehensive list, I believe it covers more than 80% of the ADC use cases that I have come across in my career.
From my experience, I would ideally evaluate any ADC platform across its capabilities to meet the key use cases of –
1.??Application Delivery
?The key use cases for Application Delivery would be to provide the critical services of Application Availability, Application Performance and Application Security irrespective of the nature of applications - be it legacy, modern, or where the applications are getting deployed - be it Public Cloud, Private Cloud, Container environment.
Software Defined ADC Platform
NSX Adv LB (Avi) platform is built on a unique Software Defined architecture with a centralised Control/Management plane and a distributed data plane which is 100% based on REST API’s. The control plane component is implemented by NSX Adv LB Controllers, and the data-plane is implemented by light weight NSX Adv LB instances called Service Engines (SEs). Both Controller and SE instances could be deployed in VM or container form factors. The Controller is the source of truth for all configurations and orchestrates the Life Cycle Management of the data-plane SEs including creation, licensing, configuration, deletion etc. This architecture is a paradigm shift from the traditional ADC architecture of running the huge monolithic code base on specialised hardware, or in VM form factor with the control, management and data plane all bundled together making it highly inefficient to scale on demand.
Intelligent Server Load Balancing
The solution provides multi-cloud load balancing and consistent policy enforcement across multiple private and public cloud deployments. The LB performs intelligent application health monitoring using both Active and Passive health monitoring methods. The passive health monitoring is used to reduce the number of connections or requests sent to a server when it detects any abnormal behaviour like TCP resets, HTTP 5xx responses etc. which makes it bit more intelligent than the legacy ADCs. ?The solution supports multiple persistence methods to cater to different requirements for the applications as well as HTTP/2 applications.
Traffic Management
The powerful Layer 4 – 7 policies (HTTP Request, HTTP Response, Policy tokens) could be easily configured from the UI to meet specific application use cases of content switching, URL rewrite, header manipulation, URL redirect etc.
To meet any custom traffic management use cases, the NSX Adv LB supports data plane programmability using DataScripts leveraging Lua scripting language (examples) and control plane programmability using ControlScripts leveraging Python scripting language.
Global Server Load Balancing
The GSLB feature implements global distribution of application traffic across multiple sites based on use cases for - Active/Active, Active/Passive, Cloud bursting or Geographic proximity based traffic routing leveraging Geo Location Database (sourced from the industry leader - Maxmind). The site cookie persistence method could be used to maintain persistence to a particular site based on the application needs.
High Availability
Being a full application proxy deployed inline to the application traffic path, another key concern is the High Availability of not only the applications, but also the ADC platform itself.
For control plane high availability, the NSX Adv LB Controllers are deployed in a 3 node cluster forming a quorum for production environments. NSX Adv LB data plane implements various High Availability methods like legacy Active/Passive or Elastic HA to meet the various availability requirements of mission critical applications. Unlike the traditional load balancers, which mostly operate in Active/Passive HA with 100% resource over-provisioning, NSX Adv LB implements a more intelligent Elastic - Active/Active or N+M HA based on Layer 2 or Layer 3 scaling methods. The declarative model ensures that the critical virtual services converge to the stated HA requirement of minimum active SEs defined by the ‘Scale per Virtual Service’ configuration in SE group properties. The virtual service would automatically get placed on a new SE with enough buffer capacity in the SE group to compensate for any SE failures. The data-plane SEs could be perceived more like “Cattles” in DevOps service model making it disposable components as compared to the “Pets” service model implemented by legacy LB solutions.
Further, the SEs are designed to operate in headless mode with last known good configurations without any traffic disruption or manual intervention in the event of a control plane failure.
Performance and Scale
When it comes to meeting Application Performance SLA’s, the ADC should first address the key concerns of the application servers or the ADC’s themselves becoming the performance bottleneck. The NSX Adv LB implements SSL termination, compression, caching, TCP multiplexing etc. to offload the backend servers apart from optimising the TCP stack to enhance the application performance. The NSX Adv LB could dynamically autoscale - scale out or scale in capacity depending upon the load - CPU, memory, connections or packets per second, to ensure that the LB instances are not becoming a performance bottleneck. Unlike the legacy LB solutions, NSX Adv LB could scale capacity on demand in 2 dimensions – scaling up vertically by beefing up the resources assigned to individual LB VM instances or by horizontally scaling out to automatically add more LB VM instances. The Layer 2 scaling supports up to 4 x LB VM instances and the Layer 3 scaling supports up to 64 x LB VM instances with BGP ECMP / RHI / BFD support. With the vertical and horizontal scaling methods, the distributed ADC fabric could scale to support up-to 10Tbps of max system throughput and 10 billion concurrent connections. For applications needing very high packet handling capacity, NSX Adv LB could also leverage specialised hardware resources using DPDK, SR-IOV etc. meeting equivalent performance characteristics of hardware based LB platforms.
DNS and IPAM
NSX Adv LB DNS provider services could be leveraged to automatically update A records for the application FQDNs when the virtual services get created in NSX LB. The DNS virtual service could be used as Authoritative DNS name server for all the sub-domains delegated to it for DNS resolutions. The DNS feature implementation supports A, AAAA, NS, CNAME, SRV, MX and TXT records.
Further, the solution could integrate with 3rd party DNS providers like Infoblox, AWS Route 53, Azure DNS etc.
NSX Adv LB IPAM provider could be used to dynamically allocate IP addresses from the subnets configured to the SE vNICS and also to the Virtual Services as needed. Further, the solution could integrate with 3rd party IPAM providers like Infoblox, AWS, Openstack etc.
The custom IPAM and DNS profile options allow one to integrate with any other service provider using custom scripts.
Security
Another major use case is Application Security and NSX Adv LB could provide comprehensive security controls across multiple layers in OSI stack – Network, SSL and Application.
The network controls could be based on IP Reputation index (provided by industry leading threat analysis company - Webroot), Geo location, IP or Service Port. IP Reputation service categorises IP addresses based on the associated threat levels like - Phishing, Botnets, Spam sources, Tor proxy etc. Further, NSX Adv LB provides DDoS mitigation using various techniques and controls like TCP SYN cookies, rate limiting etc.
The intuitive UI makes it easy to implement the SSL layer control, cipher selection and ordering based on the security score, performance and compatibility.
At layer 7, the NSX Adv LB application profile is used to provide various security controls like secure cookies, HSTS, client certificate validation, L7 DDoS mitigation etc.
The HTTP security rules could be configured to control traffic based on various Layer 7 parameters.
Further, various performance thresholds and limits could be enforced per virtual service across Layer 4 to Layer 7 to mitigate DDoS attacks targeting multiple layers.
WAF
The intelligent Web Application Firewall (iWAF) forms the last line of defence to effectively mitigate any kind of application attacks including OWASP top 10 and Zero day attacks. The NSX Adv LB could enforce both Positive security using machine learning and Negative security using known attack signatures.
Enabling WAF for any application is as simple as creating a WAF policy with associated WAF profile and assigning it to the VS configuration.
The WAF profile enforces various controls around HTTP methods, content types etc. and could be configured per application.
The iWAF security pipeline has 3 major building blocks
The iWAF policy packs all the iWAF specific security controls specific to an application.
The first step in the security pipeline is to allow all known good traffic to optimise WAF performance. Allow lists could be used to bypass WAF inspection for trusted traffic from specific IP addresses, GET requests to fetch static contents etc.
Positive security rules correspond to acceptable application behaviour patterns and is learned over a period of time using machine learning algorithms. The learning engine tracks various URI’s, parameters etc. to form a baseline of normal application behaviour which could then be promoted to a positive security rule once it has seen enough traffic samples and has gained enough confidence. The positive security rules could also be manually created or through virtual patching by integrating with Dynamic Application Security Testing (DAST) scanning tools like OWASP Zap or Qualys. The learning is a continuous process to refine the policy based on application changes, and all application traffic that conforms to the positive security rule would be accepted.
The application traffic that do not conform to allow-list or positive security rules are further processed by the security pipeline to mitigate any known attack vectors using application specific rules and CRS security rules . The application specific rules sourced from Trustwave (a leader in application specific protection) contain over 5000 different applications and frameworks to chose from and provides comprehensive security for most of the commonly used applications mitigating attacks targeting known application vulnerabilities (CVEs).
The OWASP top 10 security signatures are based on Open Source Core Rule Set (CRS) which is refined and converted to NSX Adv LB format providing protection against a multitude of known attack vectors like -SQL injection, Cross-Site-Scripting (XSS), Remote Code Execution etc. Further, custom security rules / signatures could be configured for specific applications, if needed.
The Paranoia level in the WAF policy could be configured based on the risk profile of the application to include or exclude additional rules for evaluation.
The policy could be implemented initially in detection mode for learning and then promoted to enforcement mode for mitigation based on the confidence levels of the rules configured / learned.
Security Services
Avi Pulse is a cloud based service hosted by VMware which could be optionally leveraged for updating WAF CRS signatures, IP Reputation DB, Application specific rules etc.
The application specific rules are updated daily and the IP reputation services are updated hourly (sync interval could be configured between 2-60mins) through the Avi Pulse security services. BOT detection, classification and management is currently under tech preview for version 21.1.1.
2.??Multi-Cloud
The second key use case is to support consistent policy enforcement and application service delivery in multi-cloud environments.
Private and Public Clouds
NSX Adv LB control plane natively integrates with all major infrastructure orchestrators to seamlessly automate the Life Cycle Management of data plane SE instances. This allows for consistent policy enforcement across the distributed application landscape spanning private clouds, public clouds and container environments.
In Private clouds, NSX Adv LB seamlessly integrates with VMware Software Defined Data Center (SDDC) ecosystem like vCenter, NSX-T, VCF, TKG, Horizon to name a few as well as Openstack and Linux Bare Metal deployments.
In Public clouds, NSX Adv LB seamlessly integrates with all major public cloud providers like AWS, VMC, Azure, GCP etc.
For environments where there is no native integration possible, a ‘No Orchestrator’ / Linux Server cloud could be used for manual life cycle management of the SE instances. Further, these manual tasks could be automated using any automation tools consuming the NSX Adv LB Northbound REST API’s.
领英推荐
Container Ingress
To provide secure container ingress services for modern applications deployed in micro-service based container environments, the NSX Adv LB implements an ingress controller called Avi Kubernetes Operator (AKO) which runs as a pod in the Kubernetes or Openshift container environments. The AKO pod listens to the Kubernetes control plane and automates the creation of corresponding virtual service objects in NSX Adv LB by translating the Kubernetes objects to corresponding NSX Adv LB object API’s. The NSX Adv LB controller then creates the virtual service objects and configures the data-path to enable traffic routing. The data plane SE VM instances could run in any of the IaaS platforms outside the container clusters. The updates made to the Kubernetes ingress objects are synchronised to NSX Adv LB controller by AKO. Avi Multi-Cluster Kubernetes Operator (AMKO) is a separate pod that could be deployed to provide GSLB services across multiple Kubernetes / Openshift clusters. The solution could be further extended to support Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS) and Microsoft Azure Kubernetes Service (AKS) deployments.
?
3.??Operations
The third key functionality for any ADC is to provide simplified operations, be it supporting granular Role Based Access Controls (RBAC) or providing visibility for pro-active issue resolution or enabling DevOps driven automation capabilities.
Advanced Analytics
When compared to the competition, the NSX Adv LB makes a great impression in providing visibility with its advanced analytics capabilities. The data-plane SE’s track more than 700 real-time metrics and send the telemetry data to the controllers which then performs the analytics function and provide deep insights to each transaction.
The analytics dashboard provides the colour coded health scores for virtual services, pools and the associated SE’s which could help in quickly identifying objects having issues. ?The Health Score is derived by taking into account of the performance metrics signifying end user performance times, resource penalty that signifies resource issues, anomaly penalty which signifies any unexpected traffic patterns and security penalty that signifies any active attacks, weak SSL scores etc.
The health score could be further drilled down to identify the reasons impacting the health scores.
The security analytics pane give visibility to the SSL usage patterns, security score, attacks etc.
The application analytics pane has charts that display statistics on end-to-end application response timing, throughput, open connections, connections per second, requests per second etc. The end user response times measure client RTT, server RTT, time taken by the server to respond, data transfer times to identify the performance bottlenecks. The overall application performance health score is based on the industry standard Apdex rating for measuring end user experience.
The Anomaly overlay in each chart indicates any abnormal behaviour observed when compared to the baseline data learned over a period of time.
The client application logs provide deep insights into application response times, response code, time taken across various phases of WAF rule evaluation, WAF rule hit, client inputs, header information, errors, etc.
With the elastic search function using keyword search, it becomes easy to identify the transactions having issues from millions of logs. The WAF policy rules could be fine tuned directly from the logs to simplify WAF operations. Further, the logs could be locally stored or streamed to any external logging / monitoring / reporting system like syslog, Splunk, Grafana, Prometheus etc.
Events and Alerts
To help with troubleshooting, NSX Adv LB generates events to provide a history of changes that have occurred, and default alerts are generated based on the critical events. The events and alerts could be viewed within the context of specific objects like virtual services, pools, SE etc.
Custom pro-active alerts could be configured from the events or based on 200+ metrics to notify the operations team via e-mail, Syslog, SNMP trap or execute a ControlScript to take some action like making some configuration changes.
Packet Captures
To troubleshoot data-path issues, one could easily perform packet captures from the UI and download them for further analysis. The packet capture function is intelligent enough to capture all traffic related to the virtual service irrespective of the VS being scaled out to 2 or more SEs.
Proactive Tech Support Service
Avi Pulse Services could be optionally leveraged for pro-active tech-support case creation when a critical event occurs. Without any manual intervention, the controller would automatically collect all the relevant logs and attach them to the support case created.
Multi-Tenancy
NSX Adv LB supports multi-tenancy uses cases by implementing tenancy with granular RBAC controls as well as isolation of data-plane SE’s using SE groups and VRFs. Further, labels could be tagged to objects and used as filters to enforce very granular access controls over individual objects.
DevOps
Being a Software Defined ADC platform, any GUI click or CLI command translates to a REST API call in the backend as every configurable parameter has a REST API endpoint. For any configuration, the equivalent API call could be easily derived using the browser developer tools or using the CLI command to print the same.
Further, the NSX Adv LB packs Swagger for complete API documentation specification with examples, object model definitions and an option to try the API call which makes life easier for the DevOps teams.
In short, with the automation capabilities, the entire distributed ADC fabric could be provisioned and managed using Infrastructure as Code. The Dev teams relying on imperative automation could consume the REST API or available SDK’s (Python, Java, Go etc.) and the DevOps team relying on more declarative automation could consume - Macro API, automation tools like Ansible modules and Roles, Terraform provider, vRO/vRA etc.
Flexible Upgrades
The Flexible Upgrade process ensures that the control plane and data plane could be upgraded independently. The upgrade process is non disruptive for all the virtual services deployed in HA. The SE groups mapped to different tenants could be upgraded during different maintenance windows based on each tenant’s change management process and timelines.
Conclusion
To summarise, I would like to list the top differentiating capabilities of NSX Adv LB solution that are highly relevant in any modern enterprise, especially when they are planning for the legacy LB upgrades or deploy multi-cloud application services.
1.????Software Defined ADC fabric with centralised control plane and distributed data plane to enable consistent policy enforcement for multi-cloud application services – Availability, Performance and Security.
2.????Elastic HA with auto-scale capabilities to meet the availability and scale requirements of modern applications.
3.????Advanced analytics with actionable intelligence to pro-actively mitigate potential performance bottlenecks and issues.
4.????Multi-tenancy with granular object level RBAC controls.
5.????DevOps friendly REST API based platform to automate and implement Infrastructure as Code.
I hope my perspective was useful in providing a glimpse on the possibilities of leveraging the NSX Advanced Load Balancing (Avi) solution for your application needs.
?THANK YOU…!
Note: The UI screenshots are based on NSX Adv LB version 20.1.5
Follow the links below to learn more -
Availability?
Performance
Security
Programmability
?Services
Private Cloud
?Public Cloud
Container
?Operations
?
?
?
?
?
?