In computing, you’ll do something similar. You receive many requests on your website, which puts a lot of strain on it. There’s nothing unusual about having a website. If no one visits your website, there is no point in having one.

You run into problems when your server is overloaded with people turning on their appliances. At this point, things can go wrong; if too many people visit your site, your performance will suffer. Slowly, as the number of users increases, it will become unusable. That’s not what you wanted.

The solution to this problem lies in more resources. The choice between scaling up and scaling out depends on whether you want to replace your current server with a larger one or add another smaller one.

The scaling-up process

Scaling up is quite common when an application needs more power. The database may be too large to fit in memory, the disks are full, or that more requests are causing the database to require more processing power.

Scaling up is generally easy because databases have historically had severe problems when run on multiple computers. If you try to make things work on various machines, they fail. What is the best method for sharing tables between machines? This problem has led to the development of several new databases, such as MongoDB and CouchDB.

However, it can be pretty expensive to scale up. A server’s price usually increases when you reach a particular specification. A new type of processor (that looks and performs like the previous one but costs much more than the old one) comes with this machine, a high-spec RAID controller, and enterprise-grade disks. Scaling up might be cheaper than scaling out if you upgrade components, but you’ll most likely get less bang for your buck this way. Nevertheless, if you need a couple of extra gigabytes of RAM or more disk space, or if you just want to boost the performance of a particular program, this might be the best option.

Scaling Out

Scaling out refers to having more than one machine. Scaling up has the disadvantage that you eventually reach an insurmountable limit. A single machine can’t hold all the processing power and memory it needs. If you need more, what happens?

If you have a lot of visitors, people will say you’re in an envious position if a single machine can’t handle the load. As strange as it may sound, this is a good problem! Scaling out means you can add machines as you go. You’ll run out of space and power at some point, but scaling out will undoubtedly provide more computing power than scaling up.

Scaling out also means having more machines. Therefore, if one machine fails, other machines can still carry the load. Whenever you scale up, if one machine fails, it affects everything else.

There is one big problem with scaling out. You have three machines and a single cohesive website or web application. How can you make the three machines work together to give the impression of one machine? It’s all about load balancing!

Finally, load balancing

Now, let’s get back to load balancing. The biggest challenge in load balancing is making many resources appear as one. How can you make three servers look and feel like a single website to the customer?

How does the Web work?

This journey begins with an examination of how the Web functions. Under the covers of your browser, what happens when you click Go? The book goes into great detail, even briefly discussing the TCP (Transmission Control Protocol) layer. While someone might be able to make an awe-inspiring web application, they may not be as familiar with the lower-level details that make it all function. Fortunately, this isn’t an issue since kickass software doesn’t require knowledge of the Internet’s inner workings. It would be best if you had a much better understanding of how it works to make your software scream past the competition quickly.

Lack of Visibility

Existing service provider challenges include a lack of network visibility into customer traffic. They are often unaware of the granular details of traffic profiles, leading them to over-provision bandwidth and link resilience. There are a vast amount of over-provisioned networks. Upgrades at a packet and optical layer occur without complete traffic visibility and justification. Many core networks are left at half capacity, just in a spike. Money is wasted on underutilization that could be spent on product and service innovation. You might need the analytical information for many reasons, not just bandwidth provisioning. 

Network Analytics 

Popular network analytic capability tools are sFlow and NetFlow. Nodes capture and send sFlow information to a sFlow collector, where the operator can analyze it with the sFlow collector’s graphing and analytical tools. An additional tool that can be used is a centralized SDN controller, such as an SD-WAN Overlay, that can analyze the results and make necessary changes to the network programmatically. A centralized global viewpoint enabling load balancing can aid in intelligent multi-domain Traffic Engineering (TE) decisions.

Before you proceed, you may find the following posts of interest:

1. Transport SDN
2. What Does SDN Mean
3. Load Balancer Scaling
4. Network Traffic Engineering
5. Application Delivery Architecture

Back to basics with load balancing

One use case for load balancers to solve is availability. At some stage in time, machine failure happens. This is 100%. Therefore, you should avoid single points of failure whenever feasible. This signifies that machines should have replicas. In the case of front-end web servers, there should be at least two. When you have replicas of servers, a machine loss is not a total failure of your application. Therefore, your customer should notice as little during a machine failure event as possible.

Benefits of Load Balancing:

Load balancing offers several critical benefits in modern computing:

1. Improved Scalability: By distributing client requests across multiple servers, load balancers enable systems to handle increased traffic without compromising performance. This scalability is essential for websites and applications that experience high user demand or rapid growth.

2. Enhanced Reliability: Load balancers provide fault tolerance by automatically rerouting traffic to healthy servers in case of failures or outages. This redundancy ensures that users can access the system even if individual servers go offline, minimizing service disruptions.

3. Optimal Resource Utilization: By evenly distributing traffic, load balancers ensure that server resources are utilized efficiently. This results in better overall system performance and cost-effectiveness, as organizations can make the most of their server infrastructure without unnecessary overprovisioning.

4. Geographic Load Balancing: With the increasing globalization of internet services, load balancers can distribute traffic based on geographic locations. This approach helps reduce latency and provides users faster response times, irrespective of location.

Load Balancing Techniques:

Several load-balancing techniques are commonly used in modern computing environments. These include:

1. Round Robin: In this method, requests are distributed sequentially to each server in a cyclic manner. Although simple, round-robin can be less effective if servers have different capabilities or workload capacities.

2. Least Connection: This technique directs client requests to the server with the fewest active connections. By dynamically considering server capacity, this method ensures that incoming traffic is distributed more evenly.

3. Weighted Round Robin: Weighted round robin assigns varying weights to servers based on their capabilities. Servers with higher capacities receive more traffic, enabling organizations to optimize resource utilization.

4. IP Hash: This technique consistently routes requests to the same server using the client’s IP address. IP hash helps maintain session persistence, ensuring that a client’s data remains on the same server throughout their session.

Load Balancing and Traffic Engineering

We need network traffic engineering for load balancing that allows packets to be forwarded over non-shortest paths. Tools such as Resource Reservation Protocol (RSVP) and Fast Re-Route (FRR) enhance the behavior of TE. IGP-based TE uses a distributed routing protocol to discover the topology and run algorithms to find the shortest path. MPLS/RSVP-TE enhances standard TE and allows more granular forwarding control and the ability to differentiate traffic types for CoS/QoS purposes.

Constrained Shortest Path First

The shortest path algorithm called Constrained Shortest Path First (CSPF) provides label switch paths (LSP) to take any available path in the network. The MPLS control plane is distributed and requires a distributed IGP and label allocation protocol. The question is whether a centralized controller can solve existing traffic engineering problems. It will undoubtedly make orchestrating a network more manageable.

The contents of a TED have IGP scope domain visibility. Specific applications for TE purposes require domain-wide visibility to make optimal TE decisions. The IETF has defined the Path Computation Element (PCE) used to compute end-to-end TE paths.

Link and TE attributes are shared with external components. Juniper’s SD-WAN product, NorthStar, adopts these technologies and promises network-wide visibility and enhanced TE capabilities. 

Load Balancing with NorthStar SD-WAN controller

NorthStar is a new SD-WAN product by Juniper aimed at Service Providers and large enterprises that follow the service provider model. It is geared for the extensive network that owns Layer 2 links. NorthStar is an SD-WAN Path Computation Engine (PCE), defined in RFC 5440, that learns network state by Path Computation Element Protocol (PCEP).

It provides centralized control for path computation and TE purposes, enabling you to run your network more optimally. In addition, NorthStar gives you a programmable network with global visibility. It allowed you to spot problems and deploy granular control over traffic.

They provide a simulation environment where they learn about all the traffic flows on the network. This allows you to simulate what “might” happen in specific scenarios. With a centralized view of the network, they can optimize flows throughout it, enabling a perfectly engineered and optimized network.

The controller can find the extra and unused capacity, allowing the optimization of underutilized spots in the network. The analytics provided is helpful for forecasting and capacity planning. It has an offline capability, providing offline versions of your network with all its traffic flows.

It takes inputs from:

1. The network determines the topology and views link attributes.
2. Human operators.
3. Requests by Northbound REST API.

These inputs decide TE capabilities and where to place TE LSP in the network. In addition, it can modify LSP and create new ones, optimizing the network traffic engineering capabilities.

Understand network topology

Traditional networks commonly run IGP and build topology tables. It can get overly complicated when you have a multi-area or multi-IGP running on the network. For network-wide visibility, NorthStar recommends BGP-LS. BGP-LS enables routers to export the contents of the TE database to BGP. It uses a new address family, allowing BGP to carry node and link attributes (metric, max amount of bandwidth, admin groups, and affinity bits) related to TE. BGP-LS can be used between different regions.

As its base is BGP, you can use scalable and high-availability features, such as route reflection, to design your BGP-LS network. While BGP is very scalable, its main advantage is reduced network complexity.

While NorthStar can peer with existing IGP (OSPF and ISIS), BGP-LS is preferred. Knowing the topology and attributes, the controller can set up LSP; for example, if you want a diverse LSP, it can perform a diverse LSP path computation. 

LSP & PCEP

There are three main types of LSPs in a NorthStar WAN-controlled network:

1. A Vanilla-type LSP. It is a standard LSP, configured on the ingress router and signaled by RSVP.
2. A delegated LSP is configured on the ingress router and then delegated to the controller, which is authorized to make changes to this LSP.
3. The controller initiates the third LSP via a human GUI or Northbound API operation.

PCEP (Path Computation Elements Protocol) communicates between all nodes and the controller. It is used to set up and modify LSP and enable dynamic and inter-area, inter-domain traffic, and engineered path setup. It consists of two entities, PCE and PCC. Path Computation Client (PCC) and Path Computation Element (PCE) get established over TCP.

Once the session is established, PCE builds the topology database (TED) using the underlying IGP or BGP-LS. BGP-LS has enhanced TLV capabilities that have been added for PCE to learn and build this database. RSVP is still used to signal the LSP.

As the demand for fast and reliable web services grows, load balancing has become an essential component of modern computing infrastructure. By evenly distributing incoming network traffic across multiple servers, load balancers enhance scalability, reliability, and resource utilization. With various load-balancing techniques, organizations can choose the most suitable method to optimize their system’s performance and deliver an exceptional user experience. Embracing load balancing is vital for businesses seeking to stay competitive in today’s digital landscape.