Building a highspeed and scalable NetScaler solution to handle millions of requests per second

NetScalers are used by some of the world's largest service providers to handle inbound application traffic. In this article, I will show how you can deploy an NetScaler environment that will scale to millions of HTTP requests and SSL handshakes per second. In the next, I will show you how to automate that environment with a basic introduction to Terraform.

The suggested architecture will allow for multiple datacenters. In addition, it will provide load balancing, SSL offload, and the capability to secure and optimize your web application traffic. The architecture will also be scalable and fully automatable.

We will begin by reviewing technologies to get traffic to your datacenter, how to distribute that traffic across NetScalers once it arrives, and how you can configure your NetScalers with code using in-house deployment scripts.

Assuming you have sized the rest of your environment appropriately, this design should allow you to process around 100 million HTTP requests per second at each datacenter. However, the layout is scalable and can be used for smaller implementations without significant changes.

A standardized repeatable design at each datacenter

With traffic now arriving at the most appropriate datacenter, we can focus on handling the resultant traffic.  

We will design each datacenter to operate with a standardized design so that it's simple and easy to update and maintain.

We will use physical hardware NetScalers known as "MPXs" for their high performance. However, our design is equally suitable for virtual NetScalers "VPX" if you favor a complete infrastructure-as-code approach.

In our design, traffic (directed by CADS) will enter the network at the router/firewall. The router will then distribute these inbound connections to our NetScalers.

Next, the NetScalers will terminate the connections, process the requests, and deliver them to backend servers.

Finally, the servers will service the requests, potentially using local data or a storage solution replicated between your datacenters.

All of the configuration will be automated using Terraform (covered in the next article). However, the use of Terraform is a personal preference, and NetScaler also supports a range of other automation technologies, such as:

• Ansible
• Citrix ADM Service
• Nitro REST API to allow for custom scripts.

We assume you have appropriately sized your routers, firewalls, and storage. And you have ensured your storage is replicated between datacenters if your servers so require. This article shows a single router as a simplification.

Splitting traffic within the datacenter

Within the datacenter, the router/firewall will divide connections between our NetScalers using dynamic routing.  We will be automating the configuration in the next article but, it would be helpful to first understand how this could be achieved manually.

In our dynamic routing configuration, each NetScalers "announces" the IP addresses of the virtual servers (vServers) they host to the router. As each NetScaler will host vServers with the same IP addresses, the router will distribute the inbound connections between the NetScalers.

On a Cisco router, the distribution of traffic between OSPF neighbors will likely occur in hardware using a technology known as CEF, Cisco Express Forwarding, to reduce the impact on CPU. You must take care to ensure your router has sufficient resources to handle the distribution.

Additionally, to ensure all packets from a user are directed to the same NetScaler, you should consult your routers manual to ensure it is using per-destination load-sharing, which is the default on a Cisco router.

To implement our dynamic routing configuration, we first need to configure our router to have an OSFP neighbor relationship with the NetScalers. On a Cisco router, a very simplistic OSPF configuration would look as follows:

After we enter the "network 192.168.1.0 0.0.0.255 area 0" command, our router will enable OSPF on all of its interface addresses within that network range.

Next, on our NetScalers, we need to enable the OSPF routing facility with the command "enable ns feature ospf".After enabling OSPF, we can configure the NetScaler's included routing platform known as "ZebOS", by entering the command "vtysh".

The ZebOS configuration on each NetScaler would look as follows:

With the OSPF configuration created, we can tell each NetScaler to advertise the IP address of a vServer (such Load Balancing vServer) using the following command:

This command will tell the NetScaler to announce the IP address 10.0.0.10 to the router.

With each of our NetScalers now announcing itself to our router as a path to the Load Balancing vServer, the router will distribute inbound traffic to each of the NetScalers.

After enabling OSPF, you can validate communication between the NetScalers and router using the following commands on the router. Here, we can see three NetScalers on "192.168.1.10", "1.11", and "1.12" advertising the LB vServer on "10.0.0.10".

We have used a relatively simplistic configuration with a single router and each NetScaler acting as an OSPF path. As routers are generally limited to 16 OSPF paths, this design will likely be limited to 16 NetScalers.  More advanced implementations will be able to expand on this simple design.

16 low-end appliances (such as the MPX9130) will equate to around 33 million L7 HTTP requests per second. Using high-end appliances (such as the MPX26200-50S), this design should service over 100 million L7 requests/sec at each datacenter.

Note: You will have multiple routers in a production environment to meet redundancy and throughput requirements and may need to work with your network team to adjust the proposed configuration.