dns load balancing failover

GTM Load Balancer

GTM Load Balancer

In today's digital world, website performance and availability are crucial for online businesses. With increasing traffic and the need for seamless user experience, load-balancing solutions have become essential. One such solution is the Global Traffic Manager (GTM) Load Balancer, which offers advanced features to optimize website performance and ensure high availability. In this blog post, we will explore the benefits and functionalities of the GTM Load Balancer and the LTM Load Balancer.

The GTM Load Balancer is a powerful tool that distributes incoming network traffic across multiple servers or data centers. It acts as a traffic manager, ensuring each user request is directed to the most efficient server based on server health, proximity, and capacity. This helps distribute the load evenly, improves response time, and minimizes downtime.

Table of Contents

Highlights: GTM Load Balancer

The Role of a GTM Load Balancer

A GTM Load Balancer is a solution that efficiently distributes traffic across multiple web applications and services. In addition, it distributes traffic across various nodes, allowing for high availability and scalability. As a result, these load balancers enable organizations to improve website performance, reduce costs associated with hardware, and allow seamless scaling as application demand increases. It acts as a virtual traffic cop, ensuring incoming requests are routed to the most appropriate server or data center based on predefined rules and algorithms.

The Role of an LTM Load Balancer

The LTM Load Balancer, short for Local Traffic Manager Load Balancer, is a software-based solution that distributes incoming requests across multiple servers, ensuring efficient utilization of resources and preventing any single server from being overwhelmed. By intelligently distributing traffic, the LTM Load Balancer ensures high availability, scalability, and improved performance for applications and services.

Continuously Monitors

GTM Load Balancers continuously monitor server health, network conditions, and application performance. They use this information to distribute incoming traffic intelligently, ensuring that each server or data center operates optimally. By spreading the load across multiple servers, GTM Load Balancers prevent any single server from becoming overwhelmed, thus minimizing the risk of downtime or performance degradation.

Traffic Patterns

GTM Load Balancers are designed to handle a variety of traffic patterns, such as round robin, least connections, and weighted least connections. It can also be configured to use dynamic server selection, allowing for high flexibility and scalability. GTM Load Balancers work with HTTP, HTTPS, TCP, and UDP protocols, which are well-suited to handle various applications and services.

GTM Load Balancers can be deployed in public, private, and hybrid cloud environments, making them a flexible and cost-effective solution for businesses of all sizes. GTM Load Balancers have advanced features such as automatic failover, health checks, and SSL acceleration.

Related: Both of you proceed. You may find the following helpful information:

  1. DNS Security Solutions
  2. OpenShift SDN
  3. ASA Failover
  4. Load Balancing and Scalability
  5. Data Center Failover
  6. Application Delivery Architecture
  7. Port 179
  8. Full Proxy
  9. Load Balancing



GTM Load Balancing.

Key GTM Load Balancer Discussion Points:


  • Introduction to load balancing with the GTM.

  • Discussion on DNS and how it works.

  • Discussion on the DNS TTL and how this may effect load balancing.

  • Highlighting DNS pinning and cache poisoning.

  • Load balancing methods.

  • A final note on Anycast.

 

Back to Basics: GTM load balancer

Load Balancing Options

The Local Traffic Managers (LTM) and Enterprise Load Balancers (ELB) provide load-balancing services between two or more servers/applications in case of a local system failure. Global Traffic Managers (GTM) provide load-balancing services between two or more sites or geographic locations.

Local Traffic Managers, or Load Balancers, are devices or software applications that distribute incoming network traffic across multiple servers, applications, or network resources. They act as intermediaries between users and the servers or resources they are trying to access. By intelligently distributing traffic, LTMs help prevent server overload, minimize downtime, and improve system performance.

GTM Load Balancer

Main GTM Load Balaner Components

GTM Load Balancer

  • The GTM provides load-balancing services between two or more sites or geographic locations

  • GTM Load Balancers work with HTTP, HTTPS, TCP, and UDP protocols.

  • The GTM load balancer offers intelligent Domain Name System (DNS) resolution capability.

  • For security, it can enable DNS security designs and act as the authoritative DNS server or secondary authoritative DNS server web.

GTM and LTM Components

Before diving into the communication between GTM and LTM, let’s understand what each component does.

GTM, or Global Traffic Manager, is a robust DNS-based load-balancing solution that distributes incoming network traffic across multiple servers in different geographical regions. Its primary objective is to ensure high availability, scalability, and optimal performance by directing users to the most suitable server based on various factors such as geographic location, server health, and network conditions.

On the other hand, LTM, or Local Traffic Manager, is responsible for managing network traffic at the application layer. It works within a local data center or a specific geographic region, balancing the load across servers, optimizing performance, and ensuring secure connections.

As mentioned earlier, the biggest difference between the GTM and LTM is traffic doesn’t flow through the GTM to your servers.

  • GTM (Global Traffic Manager )

The GTM load balancer balances traffic between application servers across Data Centers. Using F5’s iQuery protocol for communication with other BIGIP F5 devices, GTM acts as an “Intelligent DNS” server, handling DNS resolutions based on intelligent monitors. The service determines where to resolve traffic requests among multiple data center infrastructures.

  • LTM (Local Traffic Manager)

LTM balances servers as well as caches, compresses, persists, etc. The LTM network acts as a full reverse proxy, handling client connections. The F5 LTM uses Virtual Services (VSs) and Virtual IPs (VIPs) to configure a load-balancing setup for a service.

Two load balancing methods are available with LTMs, nPath configuration and Secure Network Address Translation (SNAT). As well as load balancing, LTM performs caching, compression, persistence, and other functions.

Load Balancing

Global Traffic Manager

Load Balancing Functions

  • The GTM is an intelligent name resolver, intelligently resolving names to IP addresses. The GTM works across data center.

  • Once the GTM provides you with an IP to route to you’re done with the GTM until you ask it to resolve another name for you.

  • Similar to a usual DNS server, the GTM does not provide any port information in its resolution.

Load Balancing

Local Traffic Manager 

Load Balancing Functions

  • The LTM doesn’t do any name resolution and assumes a DNS decision has already been made.

  • When traffic is directed to the LTM traffic flows directly through its’ full proxy architecture to the servers it’s load balancing.

  • Since the LTM is a full proxy it’s easy for it to listen on one port but direct traffic to multiple hosts listening on any port specified.

Communication between GTM and LTM:

BIG-IP Global Traffic Manager (GTM) uses the iQuery protocol to communicate with the local big3d agent and other BIG-IP big3d agents. GTM monitors BIG-IP systems’ availability, the network paths between them, and the local DNS servers attempting to connect to them.

The communication between GTM and LTM occurs in three key stages:

1. Configuration Synchronization:

GTM and LTM communicate to synchronize their configuration settings. This includes exchanging information about the availability of different LTM instances, their capacities, and other relevant parameters. By keeping the configuration settings current, GTM can efficiently make informed decisions on distributing traffic.

2. Health Checks and Monitoring:

GTM continuously monitors the health and availability of the LTM instances by regularly sending health check requests. These health checks ensure that only healthy LTM instances are included in the load-balancing decisions. If an LTM instance becomes unresponsive or experiences issues, GTM automatically removes it from the distribution pool, optimizing the traffic flow.

3. Dynamic Traffic Distribution:

GTM distributes incoming traffic to the most suitable LTM instances based on the configuration settings and real-time health monitoring. This ensures load balancing across multiple servers, prevents overloading, and improves the overall user experience. Additionally, GTM can reroute traffic to alternative LTM instances in case of failures or high traffic volumes, enhancing resilience and minimizing downtime.

  • A key point: TCP Port 4353

LTMs and GTMs can work together or separately. Most organizations that own both modules use them together, and that’s where the real power lies.
They use a proprietary protocol called iQuery to accomplish this.

Through TCP port 4353, iQuery reports VIP availability/performance to GTMs. A GTM can then dynamically resolve VIPs that reside on an LTM. With LTMs as servers in GTM configuration, there is no need to monitor VIPs directly with application monitors since the LTM is doing that, and iQuery reports it back to the GTM.

Text

  •  iQuery protocol to communicate with the local big3d agent and other BIG-IP big3d agents

  • communication between GTM and LTM occurs in three key stages:

  •  Local Traffic Manager, is responsible for managing network traffic at the application layer

  • Through TCP port 4353, iQuery reports VIP availability/performance to GTMs

Button Text


 

The Role of DNS With Load Balancing

The GTM load balancer offers intelligent Domain Name System (DNS) resolution capability to resolve queries from different sources to different data center locations. It loads and balances DNS queries to existing recursive DNS servers and caches the response or processes the resolution. This does two main things. First, for security, it can enable DNS security designs and act as the authoritative DNS server or secondary authoritative DNS server web. It implements several security services with DNSSEC, enabling it to protect against DNS-based DDoS attacks.

DNS relies on UDP for transport, so you are also subject to UDP control plane attacks—secondly, performance. DNS load balancing failover can improve performance for load balancing your traffic to your data centers. DNS is much more graceful than Anycast and is a lightweight protocol.

gtm load balancer
Diagram: GTM and LTM load balancer. Source: Network Interview

Video: UDP Scanning

In this whiteboard session, we will address port scanning. Now. Port scanning can be performed against TCP and UDP ports. Identifying open ports on a target system is the stage that a bad actor has to carry out when understanding and defining the attack surface of a target.

These open ports correspond to the networked services running on a system. And it’s something you want to protect your network against. To test what is open on your network, we use the process of port scanning. And this can be done with a tool called Scapy.

Port Scanning: UDP and TCP
Prev 1 of 1 Next
Prev 1 of 1 Next

Back to Basic: Load Balancing.

Load balancing indicates spreading a processing load for an application over several different systems to improve overall performance in processing incoming requests. It splits the load that arrives into one server among several other devices. This can decrease the amount of processing done by the primary receiving server.

While splitting up different applications used to process a request among separate servers is usually the first step, there are several additional ways to increase your ability to split up and process loads—all for greater efficiency and performance. DNS load balancing failover, which we will discuss next, is the most straightforward way to load balance.

Benefits of GTM Load Balancer:

1. Enhanced Website Performance: By efficiently distributing traffic, GTM Load Balancer helps balance the server load, preventing any single server from being overwhelmed. This leads to improved website performance, faster response times, and reduced latency, resulting in a seamless user experience.

2. Increased Scalability: As online businesses grow, the demand for server resources increases. GTM Load Balancer allows enterprises to scale their infrastructure by adding more servers or data centers. This ensures that the website can handle increasing traffic without compromising performance.

3. Improved Availability and Redundancy: GTM Load Balancer offers high availability by continuously monitoring server health and automatically redirecting traffic away from any server experiencing issues. It can detect server failures and quickly reroute traffic to healthy servers, minimizing downtime and ensuring uninterrupted service.

4. Geolocation-based Routing: Businesses often cater to a diverse audience across different regions in a globalized world. GTM Load Balancer can intelligently route traffic based on the user’s geolocation, directing them to the nearest server or data center. This reduces latency and improves the overall user experience.

5. Traffic Steering: GTM Load Balancer allows businesses to prioritize traffic based on specific criteria. For example, it can direct high-priority traffic to servers with more resources or specific geographic locations. This ensures that critical requests are processed efficiently, meeting the needs of different user segments.

Key Features of GTM Load Balancer:

1. Geographic Load Balancing: GTM Load Balancer uses geolocation-based routing to direct users to the nearest server location. This reduces latency and ensures that users are connected to the server with the lowest network hops, resulting in faster response times.

2. Health Monitoring: The load balancer continuously monitors the health and availability of servers. If a server becomes unresponsive or experiences a high load, GTM Load Balancer automatically redirects traffic to healthy servers, minimizing service disruptions and maintaining high availability.

3. Flexible Load Balancing Algorithms: GTM Load Balancer offers a range of load balancing algorithms, including round-robin, weighted round-robin, and least connections. These algorithms enable businesses to customize the traffic distribution strategy based on their specific needs, ensuring optimal performance for different types of web applications.

Back to Basic: DNS Load Balancing Failover

DNS load balancing is the simplest form of load balancing. As for the actual load balancing, it is somewhat straightforward in how it works. It uses a direct method called round robin to distribute connections over the group of servers it knows for a specific domain. It does this sequentially. This means going first, second, third, etc.). To add DNS load balancing failover to your server, you must add multiple A records for a domain.

dns load balancing failover
Diagram: DNS load balancing. Source Imperva

GTM load balancer and LTM 

DNS load balancing failover

The GTM load balancer and the Local Traffic Manager (LTM) provide load-balancing services towards physically dispersed endpoints. Endpoints are in separate locations but logically grouped in the eyes of the GTM. For data center failover events, DNS is much more graceful than Anycast. With GTM DNS failover, end nodes are restarted (cold move) into secondary data centers with a different IP address.

As long as the DNS FQDN remains the same, new client connections are directed to the restarted hosts in the new data center. The failover is performed with a DNS change, making it a viable option for disaster recovery, disaster avoidance, and data center migration.

On the other hand, stretch clusters and active-active data centers pose a separate set of challenges. In this case, other mechanisms, such as FHRP localization and LISP, are combined with the GTM to influence ingress and egress traffic flows.

 

DNS Namespace Basics

Packets traverse the Internet using numeric IP addresses, not names, to identify communication devices. DNS was developed to map the IP address to a user-friendly name to make numeric IP addresses memorable and user-friendly. Employing memorable names instead of numerical IP addresses dates back to the early 1980s in ARPANET. Localhost files called HOSTS.txt mapped IP to names on all the ARPANET computers. The resolution was local, and any changes were implemented on all computers.

DNS basics
Diagram: DNS Basics. Source is Novell

Example: DNS Structure

This was sufficient for small networks, but with the rapid growth of networking, a hierarchical distributed model known as a DNS namespace was introduced. The database is distributed worldwide on what’s known as DNS nameservers that consist of a DNS structure. It resembles an inverted tree, with branches representing domains, zones, and subzones.

At the very top of the domain is the “root” domain, and then further down, we have Top-Level domains (TLD), such as .com or .net. and Second-Level domains (SLD), such as www.network-insight.net.

The IANA delegates management of the TLD to other organizations such as Verisign for.COM and. NET. Authoritative DNS nameservers exist for each zone. They hold information about the domain tree structure. Essentially, the name server stores the DNS records for that domain.

DNS Tree Structure

You interact with the DNS infrastructure with the process known as RESOLUTION. First, end stations request a DNS to their local DNS (LDNS). If the LDNS supports caching and has a cached response for the query, it will respond to the client’s requests.

DNS caching stores DNS queries for some time, which is specified in the DNS TTL. Caching improves DNS efficiency by reducing DNS traffic on the Internet. If the LDNS doesn’t have a cached response, it will trigger what is known as the recursive resolution process.

Next, the LDNS queries the authoritative DNS server in the “root” zones. These name servers will not have the mapping in their database but will refer the request to the appropriate TLD. The process continues, and the LDNS queries the authoritative DNS in the appropriate.COM .NET or. ORG zones. The method has many steps and is called “walking a tree.” However, it is based on a quick transport protocol (UDP) and takes only a few milliseconds.

 

DNS Load Balancing Failover Key Components

DNS TTL

Once the LDNS gets a positive result, it caches the response for some time, referenced by the DNS TTL. The DNS TTL setting is specified in the DNS response by the authoritative nameserver for that domain. Previously, an older and common TTL value for DNS was 86400 seconds (24 hours).

This meant that if there were a change of record on the DNS authoritative server, the DNS servers around the globe would not register that change for the TTL value of 86400 seconds.

This was later changed to 5 minutes for more accurate DNS results. Unfortunately, TTL in some end hosts’ browsers is 30 minutes, so if there is a failover data center event and traffic needs to move from DC1 to DC2, some ingress traffic will take time to switch to the other DC, causing long tails. 

DNS TTL
Diagram: DNS TTL. Source is Varonis

DNS pinning and DNS cache poisoning

Web browsers implement a security mechanism known as DNS pinning, where they refuse to take low TTL as there are many security concerns with low TTL settings, such as cache poisoning. Every time you read from the DNS namespace, there is potential DNS cache poisoning and a DNS reflection attack.

Because of this, all browser companies ignored low TTL and implemented their aging mechanism, which is about 10 minutes.

In addition, there are embedded applications that carry out a DNS lookup only once when you start the application, for example, a Facebook client on your phone. During data center failover events, this may cause a very long tail, and some sessions may time out.

DNS Packet Capture1

GTM Load Balancer and GTM Listeners

The first step is to configure GTM Listeners. A listener is a DNS object that processes DNS queries. It is configured with an IP address and listens to traffic destined to that address on port 53, the standard DNS port. It can respond to DNS queries with accelerated DNS resolution or GTM intelligent DNS resolution.

GTM intelligent Resolution is also known as Global Server Load Balancing (GSLB) and is just one of the ways you can get GTM to resolve DNS queries. It monitors a lot of conditions to determine the best response.

The GTM monitors LTM and other GTMs with a proprietary protocol called IQUERY. IQUERY is configured with the bigip_add utility. It’s a script that exchanges SSL certificates with remote BIG-IP systems. Both systems must be configured to allow port 22 on their respective self-IPs.

The GTM allows you to group virtual servers, one from each data center, into a pool. These pools are then grouped into a larger object known as a Wide IP, which maps the FQDN to a set of virtual servers. The Wide IP may contain Wild cards.

F5 GTM

Load Balancing Methods

When the GTM receives a DNS query that matches the Wide IP, it selects the virtual server and sends back the response. Several load balancing methods (Static and Dynamic) are used to select the pool; the default is round-robin. Static load balancing includes round-robin, ratio, global availability, static persists, drop packets, topology, fallback IP, and return to DNS.

Dynamic load balancing includes round trip time, completion time, hops, least connections, packet rate, QoS, and kilobytes per second. Both methods involve predefined configurations, but dynamic considers real-time events.

For example, topology load balancing allows you to select a DNS query response based on geolocation information. Queries are resolved based on the resource’s physical proximity, such as LDNS country, continent, or user-defined fields. It uses an IP geolocation database to help make the decisions. It helps service users with correct weather and news based on location. All this configuration is carried out with Topology Records (TR).

 Anycast and GTM DNS for DC failover

Anycast means you advertise the same address from multiple locations. It is a viable option when data centers are geographically far apart. Anycast solves the DNS problem, but we also have a routing plane to consider. Getting people to another DC with Anycast can take time and effort.

It’s hard to get someone to go to data center A when the routing table says go to data center B. The best approach is to change the actual routing. As a failover mechanism, Anycast is not as graceful as DNS migration with F5 GTM.

Generally, if session disruption is a viable option, go for Anycast. Web applications would be OK with some session disruption. HTTP is stateless, and it will just resend. However, other types of applications might not be so tolerant. If session disruption is not an option and graceful shutdown is needed, you must use DNS-based load balancing. Remember that you will always have long tails due to DNS pinning in browsers, and eventually, some sessions will be disrupted.

 Scale-Out Applications

The best approach is to do a fantastic scale-out application architecture. Begin with parallel application stacks in both data centers and implement global load balancing based on DNS. Start migrating users to the other data center, and when you move all the other users, you can shut down the instance in the first data center. It is much cleaner and safer to do COLD migrations. Live migrations and HOT moves (keep sessions intact) are challenging over Layer 2 links.

You need a different IP address. You don’t want to have stretched VLANs across data centers. It’s much easier to make a COLD move, change the IP, and then use DNS. The load balancer config can be synchronized to vCenter, so the load balancer definitions are updated based on vCenter VM groups.

Another reason for failures in data centers during scale-outs could be the lack of airtight sealing, otherwise known as hermetic sealing. Not having an efficient seal brings semiconductors in contact with water vapor and other harmful gases in the atmosphere. As a result, ignitors, sensors, circuits, transistors, microchips, and much more don’t get the protection they require to function correctly.

Data and Database Challenges.

The main challenge with active-active data centers and failover events is with your actual DATA and Databases. If data center A fails, how accurate will your data be? You cannot afford to lose any data if you are running a transaction database.

Resilience is achieved by storage or database-level replication that employs log shipping or distribution between two data centers with a two-phase commit. Log shipping has an RPO of non-zero, as transactions could happen a minute before. A two-phase commit synchronizes multiple copies of the database but can slow down due to latency.

GTM Load Balancer is a robust solution for optimizing website performance and ensuring high availability. With its advanced features and intelligent traffic routing capabilities, businesses can enhance their online presence, improve user experience, and handle growing traffic demands. By leveraging the power of GTM Load Balancer, online companies can stay competitive in today’s fast-paced digital landscape.

Efficient communication between GTM and LTM is essential for businesses to optimize network traffic management. By collaborating seamlessly, GTM and LTM provide enhanced performance, scalability, and high availability, ensuring a seamless experience for end-users. Leveraging this powerful duo, businesses can deliver their services reliably and efficiently, meeting the demands of today’s digital landscape.

 

Summary: GTM Load Balancer

GTM Load Balancer is a sophisticated traffic management solution that distributes incoming user requests across multiple servers or data centers. Its primary purpose is to optimize resource utilization and enhance the user experience by intelligently directing traffic to the most suitable backend server based on predefined criteria.

Key Features and Functionality

GTM Load Balancer offers a wide range of features that make it a powerful tool for traffic management. Some of its notable functionalities include:

1. Health Monitoring: GTM Load Balancer continuously monitors the health and availability of backend servers, ensuring that only healthy servers receive traffic.

2. Load Distribution Algorithms: It employs various load distribution algorithms, such as Round Robin, Least Connections, and IP Hashing, to intelligently distribute traffic based on different factors like server capacity, response time, or geographical location.

3. Geographical Load Balancing: With geolocation-based load balancing, GTM can direct users to the nearest server based on location, reducing latency and improving performance.

4. Failover and Redundancy: In case of server failure, GTM Load Balancer automatically redirects traffic to other healthy servers, ensuring high availability and minimizing downtime.

Implementation Best Practices

Implementing a GTM Load Balancer requires careful planning and configuration. Here are some best practices to consider:

1. Define Traffic Distribution Criteria: Clearly define the criteria to distribute traffic, such as server capacity, geographical location, or any specific business requirements.

2. Set Up Health Monitors: Configure health monitors to regularly check the status and availability of backend servers. This helps in avoiding directing traffic to unhealthy or overloaded servers.

3. Fine-tune Load Balancing Algorithms: Based on your specific requirements, fine-tune the load balancing algorithms to achieve optimal traffic distribution and server utilization.

4. Regularly Monitor and Evaluate: Continuously monitor the performance and effectiveness of the GTM Load Balancer, making necessary adjustments as your traffic patterns and server infrastructure evolve.

Conclusion: In a world where online presence is critical for businesses, ensuring seamless traffic distribution and optimal performance is a top priority. GTM Load Balancer is a powerful solution that offers advanced functionalities, intelligent load distribution, and enhanced availability. By effectively implementing GTM Load Balancer and following best practices, businesses can achieve a robust and scalable infrastructure that delivers an exceptional user experience, ultimately driving success in today’s digital landscape.

dns domain name system concept with big word or text and team people with modern flat style - vector illustration

DNS Structure

DNS Structure

In the vast world of the internet, the Domain Name System (DNS) plays a crucial role in translating human-readable domain names into machine-readable IP addresses. It is a fundamental component of the Internet infrastructure, enabling users to access websites and other online resources effortlessly. This blog post aims to comprehensively understand the DNS structure and its significance in the digital realm.

The DNS structure follows a hierarchical system, resembling an upside-down tree. The DNS tree structure consists of several levels. At the top level, we have the root domain, represented by a single dot (.). Below the root are top-level domains (TLDs), such as .com and .org, or country-specific ones, like .us or .uk.

Further, down the DNS hierarchy, we encounter second-level domains (SLDs) unique to a particular organization or entity. For instance, in the domain name “example.com,” “example” is the SLD.

Table of Contents

Highlights: DNS Structure

When you browse a webpage like network-insight.com, the computer needs to convert the domain name into an IP address. DNS is the protocol that accomplishes this. DNS involves queries and answers. You will make a query to resolve a web address. In response, your DNS server, typically the Active Directory server in an enterprise environment, will respond with an answer called a resource record. There are many types of DNS records and formats.

DNS happens in the background. By simply browsing www.network-insight.com, you will initiate a DNS query to resolve the IP. For example, the “A” query requests an IPv4 address for www.network-insight.com. This is the most common form of DNS request.

Endpoint Selection

Network designers are challenged with endpoint selection. How do you get eyeballs to the correct endpoint in multi-datacenter environments? Consider Domain Name System (DNS) “air traffic control” for your site. Some DNS servers should offer probing mechanisms that extract real-time data from your infrastructure for automatic traffic management—as a result, optimizing traffic management to and from the data center with efficient DNS structure, optimizing with DNS solution from GTM load balancer. Before we delve into the details of the DNS structure and the DNS hierarchy, let’s start with the basics of DNS hierarchy.

DNS Hierarchy

Considering the DNS and DNS tree structures, we have a hierarchy to manage its distributed database system. So, the DNS hierarchy, also called the domain name space, is an inverted tree structure, much like eDirectory. The DNS tree structure has a single domain at the top called the root domain. So, we have a decentralized system without any built-in security mechanism that, by default, runs over a UDP transport. Some of these called for the immediate need for DNS security solutions. Therefore, you need to keep in mind the security risks. The DNS tree structure is a large attack surface extensible and is open to many attacks, such as the DNS reflection attack.

DNS Tree Structure

The structure of the DNS is hierarchical, consisting of five distinct components.

  1. The root domain is at the apex of the domain name hierarchy. Above it are the top-level domains, further divided into second-level domains, third-level domains, and so on.
  2. The top-level domains include generic domains, such as .com, .net, and .org, and country code top-level domains, such as .uk and .us. The second-level domains are typically used to identify an organization or business. For example, the domain name google.com consists of the second-level domain Google and the top-level domain .com.
  3. Third-level domains identify a specific host or service associated with a domain name. For example, the domain name www.google.com consists of the third-level domain www, the second-level domain google, and the top-level domain .com.
  4. The fourth-level domains provide additional information about a particular host or service on the Internet. An example of a fourth-level domain is mail.google.com, which is used to access Google’s Gmail service.
  5. Finally, the fifth-level domains are typically used to identify a particular resource within a domain. An example of a fifth-level domain is docs.google.com, which is used to access Google’s online document storage service.

Domain Name System

Related: Before you proceed, you may find the following posts of interest:

  1. OpenShift SDN.
  2. SDN Data Center
  3. What is VXLAN
  4. Segment Routing
  5. SASE Model



DNS Structure

Key DNS Structure


  • Distributed database system.

  • Ride over UDP.

  • No in-built security.

  • DNS tree with Root domains,

  • Keep an eye on TTL timers.

Back to Basics With DNS Hierarchy.

Basics: DNS Structure and Process

Most of us take Internet surfing for granted. However, much is happening to make this work for you. We must consider the technology behind our simple ability to type a domain universal resource locator, aka URL, in our browsers and arrive at the landing page. The DNS structure is based on a DNS hierarchy, which makes reaching the landing page possible in seconds.

The DNS architecture consists of a hierarchical and decentralized name resolution system for resources connected to the Internet. It stores the associated information of the domain names assigned to each resource.

Thousands of DNS servers are distributed and hierarchical, but they need a complete database of all hostnames, domain names, and IP addresses. If a DNS server does not have information for a specific domain, it may have to ask other DNS servers for help. A total of 13 root name servers contain information for top-level domains such as com, net, org, biz, edu, or country-specific domains such as uk, nl, de, be, au, ca, etc.

That allows them to be reachable via the DNS resolution process. DNS queries for a resource pass through the DNS – with the URLs as parameters. Then, the DNS takes the URLs, translates them into the target IP addresses, and sends the queries to the correct resource.

1st Lab Guide: DNS Process

Domain Name System

Now that you have an idea of DNS let’s look at an example of a host that wants to find the IP address of a hostname. The host will send a DNS request and receive a DNS reply from the server. The following example shows I have a Cisco Router set up as a DNS server. I also have several public name servers configured with an external connector.

With Cisco Modelling Labs, getting external access with NAT is relatively easy. Set your connecting interface to DHCP, and the external connecter does the rest.

Note:

In the example below, the host will now send a DNS request to find the IP address of bbc.co.uk. Notice the packet capture output. Below, you can see that the DNS query uses UDP port 53. The host wants to know the IP address for bbc.co.uk. Here’s what the DNS server returns:

DNS process
Diagram: DNS Process

An administrator can query DNS name servers using TCP/IP utilities called nslookup, host, and dig. These utilities can be used for many purposes, including manually determining a host’s IP address, checking DNS resource records, and verifying name resolution.

One of Dig’s primary uses is retrieving DNS records. By querying a specific domain, Dig can provide information such as the IP address associated with the domain, mail server details, and even the DNS records’ time-to-live (TTL) value. We will explore the types of DNS records that can be queried using Dig, including A, AAAA, MX, and NS records.

Advanced Dig Techniques

Dig goes beyond simple DNS queries. It offers advanced techniques to extract more detailed information. We will uncover how to perform reverse DNS lookups, trace the DNS delegation path, and gather information about DNSSEC (Domain Name System Security Extensions). These advanced techniques can be invaluable for network administrators and security professionals.

Using Dig for Troubleshooting

Dig is a powerful troubleshooting tool that can help diagnose and resolve network-related issues. We will cover common scenarios where Dig can rescue, such as identifying DNS resolution problems, checking DNS propagation, and verifying DNSSEC signatures. 

DNS Root Servers

Understanding the Basic Syntax

Dig command follows a straightforward syntax: `dig [options] [domain] [type]`. Let’s break it down:

    • Options: Dig offers a range of options to customize your query. For example, the “+short” option provides only concise output, while the “+trace” option traces the DNS delegation path.
    • Domain: Specify the domain name you want to query. It can be a fully qualified domain name (FQDN) or an IP address.
    • Type: The type parameter defines the type of DNS record to retrieve. It can be A, AAAA, MX, NS, and more.

Exploring Advanced Functionality

Dig offers more advanced features that can enhance your troubleshooting and analysis capabilities.

    • Querying Specific DNS Servers: The “@server” option lets you query a specific DNS server directly. This can be useful for testing DNS configurations or diagnosing issues with a particular server.
    • Reverse DNS Lookup: Dig can perform reverse DNS lookups using the “-x” option followed by the IP address. This lets you obtain the domain name associated with a given IP address.

Analyzing DNSSEC Information

DNSSEC (Domain Name System Security Extensions) provides a layer of security to DNS. Dig can assist in retrieving and verifying DNSSEC-related information.

    • Checking DNSSEC Validation: The “+dnssec” option enables DNSSEC validation. Dig will fetch the DNSSEC signatures for the queried domain, allowing you to ensure the integrity and authenticity of the DNS responses.

Troubleshooting DNS Issues

Dig proves to be a valuable tool for troubleshooting DNS-related problems.

    • Checking DNS Resolution: By omitting the “type” parameter, Dig retrieves the default A record for the specified domain. This can help identify if the DNS resolution is functioning correctly.
    • Analyzing Response Times: Dig provides valuable information about response times, including the time DNS servers take to respond to queries. This can aid in identifying latency or performance issues.

Dig command syntax

DNS Architecture

DNS is a hierarchical system, with the root at the top and various levels of domains, subdomains, and records below. The Internet root server manages top-level domains such as .com, .net, and .org at the root level. These top-level domains are responsible for managing their subdomains and records.

Below the top-level domains are the authoritative nameservers, which are responsible for managing the records of the domains they are responsible for. These authoritative nameservers are the source of truth for the DNS records and are responsible for responding to DNS queries from clients.

At the DNS record level, there are various types of records, such as A (address) records, MX (mail exchange) records, and CNAME (canonical name) records. Each record type serves a different purpose and provides information about the domain or subdomain.

DNS structure
Diagram: DNS Structure. Source EIC.

Name Servers:

Name servers are the backbone of the DNS structure. They store and distribute DNS records, including IP addresses associated with domain names. When a user enters a domain name in their web browser, the browser queries the nearest name server to retrieve the corresponding IP address. Name servers are distributed globally, ensuring efficient and reliable DNS resolution.

DNS Zones:

A DNS zone refers to a specific portion of the DNS namespace managed by an authoritative name server. Zones allow administrators to control and maintain DNS records for a particular domain or subdomain. Each zone consists of resource records (RRs) that hold various types of information, such as A records (IP addresses), MX records (mail servers), CNAME records (aliases), and more.

DNS Resolution Process:

When a user requests a domain name, the DNS resolution occurs behind the scenes. The resolver, usually provided by the Internet Service Provider (ISP), starts by checking its cache for the requested domain’s IP address. If the information is not cached or has expired, the resolver sends a query to the root name servers. The root name servers respond by directing the resolver to the appropriate TLD name servers. Finally, the resolver queries the authoritative name server for the specific domain and receives the IP address.

DNS Caching:

Caching is implemented at various levels to optimize the DNS resolution process and reduce the load on name servers. Caching allows resolvers to store DNS records temporarily, speeding up subsequent requests for the same domain. However, caching introduces the challenge of ensuring timely updates to DNS records when changes occur, as outdated information may persist until the cache expires.

DNS Traffic Flow:

First, two concepts are essential to understand. Every client within an enterprise network won’t be making external DNS queries. Instead, they make requests to the local DNS server or DNS resolver, which makes the external queries on their behalf. The communication chain for DNS resolve can involve up to three other DNS servers to resolve any hostname fully. The other concept to consider is caching. Before a client requests a DNS server, it will check the local browser and system cache.

In general, DNS records are cached in three locations, and keeping these locations secured is essential. First is the browser cache, which is usually stored for a very short period. If you’ve ever had a problem with a website fixed by closing and reopening or browsing with an incognito tab, the root issue probably had something to do with the page being cached. Next is the operating system cache. It doesn’t make sense for a server to make hundreds of requests when multiple users visit the same page, so this efficiency is beneficial. However, it still presents a security risk.

2nd Lab Guide: Delving into DNS data

DNS Capture

In the lab guide, we will delve more into DNS data. Before digging into the data, it’s essential to understand some general concepts of DNS:

To browse a webpage (www.network-insight.net), the computer must convert the web address to an IP address. DNS is the protocol that accomplishes this

DNS involves queries and answers. You will make a query to resolve a web address. In response, your DNS Server (typically the Active Directory Server for an enterprise environment) will respond with an answer called a resource record. There are many types of DNS records. Notice below in the Wireshark capture, I am filtering only for DNS traffic.

In this section, you will generate some sample DNS traffic. By simply browsing www.network-insight.net, you will initiate a DNS query to resolve the IP. I have an Ubuntu host running on a VM. Notice that your first query is an “A” query, requesting an IPv4 address for www.network-insight.net. This is the most common form of DNS request.

As part of your web request, this automatically initiated two DNS queries. The second (shown here) is an “AAAA” query requesting an IPv6 address.

Note: In most instances, the “A” record response will be returned first; however, in some cases, you will see the “AAAA” response first. In either instance, these should be the 3rd and 4th packets.

Analysis:

    • The IP header contains IPv4 information. This is the communication between the host making the request (192.168.18.130) and the DNS Server (192.168.18.2). Typical DNS operates over UDP, but sometimes it works over TCP. DNS over UDP can open up some security concerns.
    • This means there’s no error checking or tracking in the network communication. Because of this, the DNS server will return a copy of the original query in the response to ensure they stay matched up.
    • Next are two A records containing the IPv4 answers. It’s pervasive for popular domains to have multiple IPs for load-balancing purposes.

Nslookup stands for “name server lookup.” It is a command-line tool for querying the Domain Name System (DNS) and obtaining information about domain names, IP addresses, and other DNS records. Nslookup is available on most operating systems and provides a simple yet powerful way to investigate DNS-related issues.

Nslookup offers a range of commands that allow users to interact with DNS servers and retrieve specific information. Some of the most commonly used commands include querying for IP addresses, performing reverse lookups, checking DNS records, and troubleshooting DNS configuration problems.

nslookup command

  1. Use the -query option to request only an ‘A’ record:
    nslookup -query=A www.network-insight.net
  2. Use the -debug option to display the full response information:
    nslookup -debug www.network-insight.net: This provides a much more detailed response, including the Time-to-Live (TTL) values and any additional record information returned.
  3. You can also perform a reverse DNS lookup by sending a Pointer Record (PTR) and the IP address:
    nslookup -type=ptr xx.xx.xx.xx

Analysis: 

The result is localhost, despite us knowing that the IP given belongs to www.network-insight.net. This is a security strategy to bypass source address checks and prevent individuals from performing PTR lookups for some domains.

Domain Name System Fundamentals

DNS Tree

The domain name system (DNS) is a naming database in which Internet domain names are located and translated into Internet Protocol (IP) addresses. It uses a hierarchy to manage its distributed database system. The DNS hierarchy, also called the domain name space, consists of a DNS tree with a single domain at the top of the structure called the root domain.

Consider DNS a naming system that is both hierarchical and distributed. Because of the hierarchical structure, you can assign the same “label” to multiple machines (for example, www.abc.com maps to 10.10.10.10 and 10.10.10.20).

DNS structure
Diagram: DNS structure

Domain Name System and its Operations

DNS servers are machines that respond to DNS queries sent by clients. Servers can translate names and IP addresses. There are differences between an authoritative DNS server and a caching server. A caching-only server is a name server with no zone files. It is not authoritative for any domain.

Caching speeds up the name-resolution process. This server can help improve a network’s performance by reducing the time it takes to resolve a hostname to its IP address. This can minimize web browsing latency, as the DNS server can quickly resolve the hostname and connect you to the website. A DNS caching server can also reduce the network’s data traffic, as DNS queries are sent only once, and the cached results are used for subsequent requests.

It can be viewed as a positive and negative element of DNS. Caching reduces the delay and number of DNS packets transmitted. On the negative side, it can produce stale records, resulting in applications connecting to invalid IP addresses and increasing the time applications failover to secondary services.

Ensuring that the DNS caching server is configured correctly is essential, as it can cause issues if the settings are incorrect. Additionally, it is crucial to ensure that the server is secure and not vulnerable to malicious attacks.

DNS Caching
Diagram: DNS Caching. Source: Bluecatnetworks.
  • A key point: Domain name system and TTL

The Time-to-Live (TTL) fields play an essential role in DNS. It controls how long a record should be stored in the cache. Choosing the suitable TTL timer per application is an important task. A short TTL can send too many queries, while a long TTL can’t capture any changes in the records.

DNS proxies and DNS resolvers respect the TTL setting for the form and usually honor TTL values as they should be. However, applications do not necessarily keep the TTL, which becomes problematic with failover events.

DNS Main Components

Main DNS Components

DNS Structure and DNS Hierarchy

  • The DNS structure follows a hierarchical system, resembling an upside-down tree.

  • We have a decentralized system without any built-in security mechanism.

  • There are various types of records at the DNS record level, such as A (address) records.

  • Name servers are the backbone of the DNS structure. They store and distribute DNS records.

  • Caching speeds up the name-resolution process. It can be viewed as a positive and negative element of DNS.

Site-selection considerations: Load balance data centers?

DNS is used to perform site selection. Multi-data centers use different IP endpoints in each data center, and DNS-based load balancing is used to send clients to one of the data centers. The design is to start using random DNS responses and slowly migrate to geo-location-based DNS load balancing. There are many load-balancing strategies, and different methods match different requirements.

3rd Lab Guide: Cloud DNS and Geolocation.

Google Cloud DNS

DNS routing policies steer traffic based on query (for example, round robin or geolocation). You can configure routing policies by creating special ResourceRecordSets (in the collection sense) with particular routing policy values.

We will examine Cloud DNS routing policies in this lab. Users can configure DNS-based traffic steering using cloud DNS routing policies. Routing policies can be divided into two types. 

Note:

  1. There are two types of routing policies: Weighted Round Robin (WRR) and Geolocation (GEO). Creating ResourceRecordSets with particular routing policy values can be used to configure routing policies.
  2. When resolving domain names, use WRR to specify different weights per ResourceRecordSet. By resolving DNS requests according to the configured weights, cloud DNS routing policies ensure traffic is distributed across multiple IP addresses.
  3. I have configured the Geolocation routing policy in this lab. Provide DNS answers corresponding to source geo locations using GEO. The geolocation routing policy applies to the nearest match if the traffic source location does not match any policy items exactly.
  4. Here, we have a Cloud DNS routing policy, create ResourceRecordSets for geo.example.com, and configure the Geolocation policy to help ensure a client request is routed to a server in the client’s closest region.

Above, we have three client VMs in the same default VPC but in different regions. There is a Europe, USA, and Asia region. There is a web server in the European region and one in the USA region. There is no web server in Asia.

I have created a firewall to allow access to the VMs. I have permitted SSH to the client VM for testing and HTTP for the webservers to accept CURL commands when we try the geolocation.

Analysis:

It’s time to test the configuration; I SSH into all the client VMs. Since all of the web server VMs are behind the geo.example.com domain, you will use cURL command to access this endpoint.

Since you are using a Geolocation policy, the expected result is that:

    • Clients in the US should always get a response from the US-East1 region.
    • The client in Europe should always get a response from the Europe-West2 region.
    • Since the TTL on the DNS record is set to 5 seconds, a sleep timer of 6 seconds has been added. The sleep timer will ensure you get an uncached DNS response for each cURL request. This command will take approximately one minute to complete.
    • When we run this test multiple times and analyze the output to see which server is responding to the request, the client should always receive a response from a server in the client’s region.

Knowledge check: DNS-based load balancing

DNS-based load balancing is an approach to distributing traffic across multiple hosts by using DNS to map requests to the appropriate host. It is a cost-effective way of scaling and balancing a web application or website load across multiple servers.

With DNS-based load balancing, each request is routed to the appropriate server based on DNS resolution. The DNS server is configured to provide multiple responses pointing to different servers hosting the same service.

The client then chooses one of the responses and sends its request to that server. The subsequent requests from the same client are sent to the same server unless the server becomes unavailable; in this case, the client will receive a different response from the DNS server and send its request to a different server.

DNS: Asynchronous Process

This approach has many advantages, such as improved reliability, scalability, and performance. It also allows for faster failover if one of the servers is down since the DNS server can quickly redirect clients to another server. Additionally, since DNS resolution is an asynchronous process, clients can receive near real-time responses and updates as servers are added or removed from the system.

DNS structure
Diagram: DNS requests

Route Health Injection (RHI)

Try to combine the site selector ( the device that monitors the data centers) with routing, such as Route Health Injection ( RHI ), to overcome the limitation of cached DNS entries. DNS is used outside of performing load distribution among data centers, and Interior Gateway Protocol (IGP) is used to reroute internal traffic to the data center.

Avoid false positives by tuning the site selector accordingly. DNS is not always the best way to fail the data center. DNS failover can quickly influence 90 % of incoming data center traffic within the first few minutes.

If you want 100% of traffic, you will probably need additional routing tricks and advertising the IP of the secondary data center with conditional route advertisements or some other form of route injection.

Domain name system
Diagram: Load balancing

Zone File Presentation

The application has changed, and the domain name system and DNS structure must be more intelligent. Users look up an “A” record for www.XYX.com, and there are two answers. When you have more than one answer, you have to think more about zone file presentation, what you offer, and based on what criteria/metrics.

Previously, the DNS structure was a viable solution with BIND. You had a primary/secondary server redundancy model with an exceedingly static configuration. People weren’t building applications with distributed data center requirements. Application requirements started to change in early 2000 with anycast DNS. DNS with anycast became more reliable and offered faster failover. Nowadays, performance is more of an issue. How quickly can you spit out an answer?

Ten years ago, to have the same application in two geographically dispersed data centers was a big deal. Now, you can spin up active-active applications in dispersed MS Azure and Amazon locations in seconds. Tapping new markets in different geographic areas takes seconds. The barriers to deploying applications in multi-data centers have changed, and we can now deploy multiple environments quickly.

Geographic routing and smarter routing decisions

Geographic routing is where you try to figure out where a user is coming from based on the Geo IP database. From this information, you can direct requests to the closest data center. Unfortunately, this doesn’t always work, and you may experience performance problems.

To make intelligent decisions, you need to take in all kinds of network telemetry about customers’ infrastructure and what is happening on the Internet now. Then, they can make more intelligent routing decisions. For this, you can analyze information about the end-user application to get an idea about what’s going on – where are you / how fast are your pipes, and what speed do you have?

The more they know, the more granular routing decisions are made. For example, are your servers overloaded, and at what point of saturation are your Internet or WAN pipes? They get this information using an API-driven approach, not dropping agents on servers.  

Geographical location – Solution

The first problem with geographical location is network performance. Geographical location is not relevant to how close things are. Second, you are looking at resolving the DNS server, not the client. You receive the IP address of the DNS resolver, not the end client’s IP address. Also, the user sometimes uses a DNS server not located where they are.

The first solution is an extension to DNS protocol – “EDNS client subnets.” This gets the DNS server to forward end-user information, including IP addresses. Google and OpenDNS will deliver the first three octets of the IP address, attempting to provide geographic routing based on the IP address of the actual end-user and not the DNS Resolver. To optimize response times or minimize packet loss, you should measure the metrics you are trying to optimize and then make a routing decision. Capture all information and then turn it into routing data.

Trying to send users to the “BEST” server varies from application to application. The word “best” really depends on the application. Some application performance depends heavily on response times. For example, streaming companies don’t care about RTT returning the first MPEG file. It depends on the application and what route you want.

 

DNS structure: DNS Security designs

DNS pinning

DNS pinning is a technique to ensure that the IP address associated with a domain name remains consistent. It involves creating an association between a domain name and the IP address of the domain’s authoritative nameserver. This association is a DNS record and is stored in a DNS database.

When DNS pinning is enabled, an organization can ensure that the IP address associated with a domain name remains the same. This is beneficial in several ways. First, it helps ensure that users are directed to the correct server when accessing a domain name. Second, it helps prevent malicious actors from hijacking a domain name and redirecting traffic to a malicious server.

DNS Spoofing

So, the main reason for DNS pinning in browsers is enabled due to security problems with DNS spoofing. Browsers that don’t honor the TTL get stuck with the same IP for up to 15 minutes. Applications should always keep the TTL for reasons mentioned at the start of the post—no notion of session stickiness with DNS. DNS has no sessions, but you can have consistent routing hashing; the same clients go to the same data center.

Route hashing optimizes cache locality. It’s like stickiness for DNS and is used for data cache locality. However, most users use the same data center based on “source IP address” or other “EDNS client subnet” information.

3rd Lab Guide: Advanced DNS

DNS Advanced Configuration

Every client within a network won’t be making external DNS queries. Instead, they make requests to the local DNS Server or DNS Resolver, and it makes the external queries on their behalf. The communication chain for DNS Resolve can involve up to three other DNS servers to fully resolve any hostname. All of which need to be secured.

DNS traffic flow

Note: The other concept to consider is caching. Before a client event requests the DNS Server, it will first check the local browser and system cache. DNS records are generally cached in three locations, and keeping these locations secured is essential.

    • First is the browser cache, which is usually stored for a very short period. If you’ve ever had a problem with a website fixed by closing/reopening or browsing with an incognito tab, the root issue probably had something to do with the page being cached.
    • Next is the Operating System (OS) cache. We will view this in the screenshot below.
    • Finally, we have the DNS Server’s cache. It doesn’t make sense for a Server to make hundreds of requests when multiple users visit the same page, so this efficiency is beneficial. However, it still presents a security risk.

Take a look at your endpoint’s DNS Server configuration. In most Unix-based systems, this is found in the resolv.conf file

Configuration options:

The resolv.conf file comprises various configuration options that determine how the DNS resolver library operates. Let’s take a closer look at some of the essential options:

1. nameserver: This option specifies the IP address of the DNS server that the resolver should use for name resolution. Multiple nameserver lines can be included to provide backup DNS servers if the primary server is unavailable.

2. domain: This option sets the default domain for the resolver. When a domain name is entered without a fully qualified (FQDN), the resolver appends the domain option to complete the FQDN.

3. search: Similar to the domain option, the search option defines a list of domains that the resolver appends to incomplete domain names. This allows for easier access to resources without specifying the complete domain name.

4. options: The options option provides additional settings such as timeout values, the order in which the resolver queries DNS servers and other resolver behaviors.

Analysis:

    • The nameserver is the IP address of the DNS server. In this case, 127.0.0.53 is listed because the “system-resolved” service is running. This service manages the DNS routing and local cache for this endpoint, which is typical for cloud-hosted endpoints. You can also have multiple nameservers listed here.
    • Options allow for certain modifications. In our example, edns0 allows for larger replies, and trust-ad is a configuration for DNSSEC and validating responses.

Now, look at your endpoint’s host file. This is a static mapping of domain names with IP addresses. Per the notice above, I have issued the command cat /etc/hosts. This file has not been modified and shows a typical configuration. If you were to send a request to localhost, an external DNS request is not necessary because a match already exists in the host’s file and will translate to 127.0.0.1.

The etc/hosts file, found in the root directory of Unix-based operating systems, is a simple text file that maps hostnames to IP addresses. It serves as a local DNS (Domain Name System) resolver, allowing the computer to bypass DNS lookup and directly associate IP addresses with specific domain names. Maintaining a record of these associations, the etc/hosts file expedites resolving domain names, improving network performance.

Finally, I modified the host’s file to redirect DNS to a fake IP address. This IP address does not exist. Notice that with the NSLookup command, DNS has been redirected to the fake IP.

Recap on DNS Tree Structure

The DNS tree structure is a hierarchical organization of domain names, starting from the root domain and branching out into top-level domains (TLDs), second-level domains, and subdomains. It resembles an inverted tree, where each node represents a domain or subdomain, and the branches represent their relationship.

Components of the DNS Tree Structure:

a) Root Domain:

The root domain is at the top of the DNS tree structure, denoted by a single dot (.). It signifies the beginning of the hierarchy and is the starting point for all DNS resolutions.

b) Top-Level Domains (TLDs):

Below the root domain are the TLDs, such as .com, .org, .net, and country-specific TLDs like .uk or .de. Different organizations manage TLDs and are responsible for specific types of websites.

c) Second-Level Domains:

After the TLDs, we have second-level domains, the primary domains individuals or organizations register. Examples of second-level domains include google.com, apple.com, or microsoft.com.

d) Subdomains:

Subdomains are additional levels within a domain. They can be used to create distinct website sections or serve specific purposes. For instance, blog.google.com or support.microsoft.com are subdomains of their respective second-level domains.

A Distributed and Hierarchical database

The DNS system is distributed and hierarchical. Although there are thousands of DNS servers, none has a complete database of all hostnames/domain names / IP addresses. DNS servers can have information for specific domains, but they may have to query other DNS servers if they do not. Thirteen root name servers store information for generic top-level domains, such as com, net, org, biz, edu, or country-specific domains, such as UK, nl, de, be, au, ca.

13 root name servers at the top of the DNS hierarchy handle top-level domain extensions. For example, a name server for .com will have information on cisco.com, but it won’t know anything about cisco.org. It will have to query a name server responsible for the org domain extension to get an answer.

For the top-level domain extensions, you will find the second-level domains. Here’s where you can find domain names like Cisco, Microsoft, etc.

Further down the tree, you can find hostnames or subdomains. For example, tools.cisco.com is the hostname of the VPS (virtual private server) that runs this website. An example of a subdomain is tools.cisco.com, where vps.tools.cisco.com could be the hostname of a server in that subdomain.

How the DNS Tree Structure Works:

When a user enters a domain name in their web browser, the DNS resolver follows a specific sequence to resolve the domain to its corresponding IP address. Here’s a simplified explanation of the process:

– The DNS resolver starts at the root domain and queries the root name servers to identify the authoritative name servers for the specific TLD.

– The resolver then queries the TLD’s name server to find the authoritative name servers for the second-level domain.

– Finally, the resolver queries the authoritative name server of the second-level domain to obtain the IP address associated with the domain.

The DNS tree structure ensures the scalability and efficient functioning of the DNS. Organizing domains hierarchically allows for easy management and delegation of authority for different parts of the DNS. Moreover, it enables faster DNS resolutions by distributing the workload across multiple name servers.

The DNS structure serves as the backbone of the internet, enabling seamless and efficient communication between users and online resources. Understanding the hierarchical nature of domain names, the role of name servers, and the DNS resolution process empowers individuals and organizations to navigate the digital landscape easily. By grasping the underlying structure of DNS, we can appreciate its significance in enabling the interconnectedness of the modern world.

Summary: DNS Structure

In today’s interconnected digital world, the Domain Name System (DNS) plays a vital role in translating domain names into IP addresses, enabling seamless communication over the internet. Understanding the intricacies of DNS structure is key to comprehending the functioning of this fundamental technology.

Section 1: What is DNS?

DNS, or the Domain Name System, is a distributed database system that converts user-friendly domain names into machine-readable IP addresses. It acts as the backbone of the internet, facilitating the efficient routing of data packets across the network.

Section 2: Components of DNS Structure

The DNS structure consists of various components working harmoniously to ensure smooth domain name resolution. These components include the root zone, top-level domains (TLDs), second-level domains (SLDs), and authoritative name servers. Each component has a specific role in the hierarchy.

Section 3: The Root Zone

At the very top of the DNS hierarchy lies the root zone. It is the starting point for all DNS queries, containing information about the authoritative name servers for each top-level domain.

Section 4: Top-Level Domains (TLDs)

Below the root zone, we find the top-level domains (TLDs). They represent the highest level in the DNS hierarchy and are classified into generic TLDs (gTLDs) and country-code TLDs (ccTLDs). Examples of gTLDs include .com, .org, and .net, while ccTLDs represent specific countries like .us, .uk, and .de.

Section 5: Second-Level Domains (SLDs)

Next in line are the second-level domains (SLDs). These are the names chosen by individuals, organizations, or businesses to create unique web addresses under a specific TLD. SLDs customize and personalize the domain name, making it more memorable for users.

Section 6: Authoritative Name Servers

Authoritative name servers store and provide DNS records for a specific domain. When a DNS query is made, the authoritative name server provides the corresponding IP address, allowing the user’s device to connect with the desired website.

Conclusion:

In conclusion, the DNS structure serves as the backbone of the internet, enabling seamless communication between devices using user-friendly domain names. Understanding the various components, such as the root zone, TLDs, SLDs, and authoritative name servers, helps demystify the functioning of DNS. By grasping the intricacies of DNS structure, we gain a deeper appreciation for the technology that powers our online experiences.

rsz_load_balancing_

Load Balancing and Scale-Out Architectures

load balancer scaling

 

Load Balancer Scaling

In the rapidly evolving world of technology, where businesses rely heavily on digital infrastructure, load balancing has become critical to ensuring optimal performance and reliability. Load balancing is a technique used to distribute incoming network traffic across multiple servers, preventing any single server from becoming overwhelmed. In this blog post, we will explore the significance of load balancing in modern computing and its role in enhancing scalability, availability, and efficiency.

One of the primary reasons why load balancing is crucial is its ability to scale resources effectively. As businesses grow and experience increased website traffic or application usage, load balancers distribute the workload evenly across multiple servers. By doing so, they ensure that each server operates within its capacity, preventing bottlenecks and enabling seamless scalability. This scalability allows businesses to handle increased traffic without compromising performance or experiencing downtime, ultimately improving the overall user experience.

Highlights: Load Balancer

  • Load Balancer Scaling

How to scale load balancer? When considering load balancer scaling and load balancer scalability, we need to recap the basics of scaling load balancers. A load balancer is a device that distributes network traffic across multiple servers. It provides an even distribution of traffic across multiple servers, so no single server is overloaded with requests. This helps to improve overall system performance and reliability. Load balancers can balance traffic between multiple web servers, application servers, and databases.

  • Geographic Locations

They can also be used to balance traffic between different geographic locations. Load balancers are typically configured to use round-robin, least connection, or source-IP affinity algorithms to determine how to distribute traffic. They can also be configured to use health checks to ensure that only healthy servers receive traffic. By distributing the load across multiple servers, the load balancer helps reduce the risk of server failure and improve overall system performance.

  • Load Balancers and the OSI Model

Load balancers operate at different Open Systems Interconnection ( OSI ) Layers from one data center to another; joint operation is between Layer 4 and Layer 7. The load balance function becomes the virtual representation of the application. Internal applications are represented by a virtual IP address ( VIP ). VIP acts as a front-end serving external clients’ requests. Data centers hosts-unique applications with different requirements. Therefore load balancing and scalability will vary depending on what applications are housed.

  • Understanding the Application

For example, every application is unique concerning the number of sockets, TCP connections ( short-lived or long-lived ), idle time-out, and activities in each session regarding packets per second. Therefore, understanding the application structure and protocols is one of the most critical elements in determining how to scale load balancer and design an effective load-balancing solution. Let us examine Direct Server Return as the first example.

 

Before you proceed, you may find the following post helpful:

  1. Auto Scaling Observability
  2. DNS Security Solutions
  3. Application Delivery Network
  4. Overlay Virtual Networking
  5. Dynamic Workload Scaling
  6. GTM Load Balancer

 



Load Balancers

Key Load Balancer Scaling Discussion Points:


  • Introduction to load balancer scaling and what is involved.

  • Highlighting the details of how to scale load balancer.

  • Critical points on load balancer scalability: Up and Out.

  • Technical details on the different load balancer scaling types: Network and Application.

  • Technical details on Layer 2 and 3 load balancing.

 

  • A key point: Video on TCP congestion control

In this video tutorial, we will address the concept of TCP Congestion control. The discrepancy and uneven bandwidth allocation for flow boil down to the natural behavior of how TCP reacts and interacts with insufficient packet buffers and the resulting packet drops. TCP congestion control refers to the set of algorithms and techniques implemented in TCP to regulate the rate at which data is transmitted over a network.

Its primary objective is to prevent network congestion, which occurs when the volume of sent data exceeds the network’s capacity to handle it effectively. Congestion control mechanisms aim to balance maximizing network utilization and minimizing packet loss, delay, and jitter.

 

Tech Brief Video Series - Networking | TCP Congestion Control
Prev 1 of 1 Next
Prev 1 of 1 Next

 

Back to basics with load balancers and load balancing

Availability:

Load balancing plays a significant role in maintaining high availability for websites and applications. By distributing traffic across multiple servers, load balancers ensure that even if one server fails, others can continue handling incoming requests. This redundancy helps to minimize downtime and ensures uninterrupted service for users. In addition, load balancers can also perform health checks on servers, automatically detecting and redirecting traffic away from malfunctioning or overloaded servers, further enhancing availability.

Efficiency:

Load balancers optimize the utilization of computing resources by intelligently distributing incoming requests based on predefined algorithms. This even distribution prevents any single server from being overwhelmed, improving overall system performance. By utilizing load balancing, businesses can ensure that their servers operate optimally, utilizing available resources and minimizing the risk of performance degradation or system failures.

Scale up and Scale-out

How is this like load balancing in the computing world? It all comes down to having finite resources and attempting to make the best potential use of them. For example, you may have the goal of making your websites fast; to do that, you must route your requests to the machines best capable of handling them. To get around this, you need more resources.

For example, you can buy a giant machine to replace your current server, known as scale-up and pricey, or another small device that works alongside your existing server, known as scale-out. As noted, the biggest challenge in load balancing is trying to make many resources appear as one. So we can have load balancing with DNS, content delivery networks, and HTTP load balancing. We also need to load balance our database and network connections.

 

  • A key point: Lab on Gateway Load Balancing Protocol (GLBP)

GLBP is running between R1 and R2. The switch is not running GLPB and is used as an interconnection point. GLBP is often used internally between access layer switches and outside the data center. It is similar in operation to HSRP and VRRP. Notice when we change the priority of R2, its role changed to Active instead of Standby.

 

Gateway Load Balancer Protocol
Diagram: Gateway Load Balancer Protocol (GLBP)

 

What is Load Balancer Scaling?

Load balancer scaling refers to the process of dynamically adjusting the resources allocated to a load balancer to meet the changing demands of an application. As the number of users accessing an application increases, load balancer scaling ensures that the incoming traffic is distributed evenly across multiple servers, preventing any single server from becoming overwhelmed.

The Benefits of Load Balancer Scaling:

1. Enhanced Performance: Load balancers distribute incoming traffic among multiple servers, improving resource utilization and response times. Load balancer scaling ensures a smooth user experience even during peak traffic by preventing any single server from overloading.

2. High Availability: Load balancers play a crucial role in maintaining high availability by intelligently distributing traffic to healthy servers. If one server fails, the load balancer automatically redirects the traffic to the remaining servers, preventing service disruption.

3. Scalability: Load balancer scaling allows applications to accommodate increased traffic without manual intervention quickly. As the server load increases, additional resources are automatically allocated to handle the extra load, ensuring that the application can scale seamlessly as per the demands.

Load Balancer Scaling Strategies:

1. Vertical Scaling: This strategy involves increasing individual servers’ resources (CPU, RAM, etc.) to handle higher traffic. While vertical scaling can provide immediate relief, it has limitations in terms of scalability and cost-effectiveness.

2. Horizontal Scaling: Horizontal scaling involves adding more servers to the application infrastructure to distribute the incoming traffic. Load balancers are critical in effectively distributing the load across multiple servers, ensuring optimal resource utilization and scalability.

3. Auto Scaling: Auto-scaling automatically adjusts the number of application instances based on predefined conditions. By monitoring various metrics like CPU utilization, network traffic, and response times, auto-scaling ensures that the application can handle increased traffic loads without manual intervention.

Best Practices for Load Balancer Scaling:

1. Monitor and Analyze: Regularly monitor your application’s and load balancer’s performance metrics to identify any bottlenecks or areas of improvement. Analyzing the data will help you make informed decisions regarding load balancer scaling.

2. Implement Redundancy: To ensure high availability, deploy multiple load balancers in different availability zones. This redundancy ensures that even if one load balancer fails, the application remains accessible through the remaining ones.

3. Regularly Test and Optimize: Conduct load testing to simulate heavy traffic scenarios and verify the performance of your load balancer scaling setup. Optimize the configuration based on the test results to ensure optimal performance.

 

  • Example: Direct Server Return. 

Direct server return (DSR) is an advanced networking technology that allows servers to send data directly to a client computer without going through an intermediary. This provides a more efficient and secure data transmission between the two, leading to faster speeds and better security.
DSR is also known as loopback, direct routing, or reverse path forwarding. It is essential in various applications, such as online gaming, streaming video, voice-over-IP (VoIP) services, and virtual private networks (VPNs).

For example, Real-Time Streaming Protocol ( RTSP ). The RTSP is an application-level network protocol for multimedia transport streams. It is used in entertainment and communications systems to control streaming media servers. With this application requirement case, the initial client connects with TCP; however, return traffic from the server can be UDP, bypassing the load balancer. For this scenario, the load-balancing method of Direct Server Return is a viable option.

DSR is an excellent choice for high-speed, secure data transmission applications. It can also be used to help reduce latency and improve reliability. For example, DSR can help reduce lag and improve game performance in online gaming.

Direct Server Return
Diagram: Direct Server Return (DRS). Source Cisco.

 

How to scale load balancer

This post will first address the different load balancer scalability options that consist of scale-up and then scale-out. The scale-out is generally the path of scaling load balancers we see today, mainly as the traffic load, control, and data plane, are spread across VMs or containers that are easy to spin up and down, commonly seen for absorbing DDoS attacks.

We will then discuss how to scale load balancer and the scalability options in the application and at a network load balancing level. We will finally address the different design options for load balancing, such as user session persistence, destination-only NAT, and persistent HTTP sessions.

Scaling a load balancer lets you adjust its performance to its workload by changing the number of nodes it contains, allowing you to scale load balancer up or down at any time to meet your traffic needs. So when considering how to scale load balancer, you must first look at the application requirements and work it out from there. What load do you expect?

 

In the diagram below, we see the following.

  • Virtual IP address: A virtual IP address is an IP address that is used to virtualize a computer’s identity on a local area network (LAN). The network address translation (NAT) form allows multiple devices to share a public IP address.
  • Load Balancer Function: The load balancer is configured to receive client requests and then route the request to the most appropriate server based on a defined algorithm.
How to scale load balancer
Diagram: How to scale load balancer and load balancer functions.

 

The primary benefit of load balancer scaling is that it provides scalability. Scalability is the ability of a networking device or application to handle organic and planned network growth. Scalability is the main advantage of load balancing, and in terms of application capacity, it increases the number of concurrent requests data centers can support. So, in summary, load balancing is the ability to distribute incoming workloads to multiple end stations based on an algorithm.

Load balancers also provide several additional features. For example, they can be configured to detect and remove unhealthy servers from the pool of available servers. They also provide SSL encryption, which can help to protect sensitive data being passed between clients and servers. Finally, they can perform other tasks like URL rewriting and content caching.

 

Load Balancing

Load Balancing Algorithm

Load Balancing Method 1

Round Robin Load Balancing

Load Balancing Method 2

Weighted Round Robin Load Balancing

Load Balancing Method 3

URL Hash Load Balancing

Load Balancing Method 4

Least Connection Method

Load Balancing Method 5

Weighted Least Connection Method

Load Balancing Method 6

Least Response Time Method

 

Load Balancing with Routers

Load Balancing is not limited to load balancer devices. Routers perform load balancing, too, with routing. Across all Cisco IOS® router platforms, load balancing is a standard feature. The router automatically activates this feature when multiple routes to a destination are in the routing table.

Routing Information Protocol (RIP), RIPv2, Enhanced Interior Gateway Routing Protocol (EIGRP), Open Shortest Path First (OSPF), and Interior Gateway Routing Protocol (IGRP) are standard routing protocols, or they are derived from static routing and packet forwarding protocols. When forwarding packets, it allows a router to use multiple paths.

  • For process-switching — load balancing is on a per-packet basis, and the asterisk (*) points to the interface over which the next packet is sent.
  • For fast-switching — load balancing is on a per-destination basis, and the asterisk (*) points to the interface over which the next destination-based flow is sent.
IOS Load Balancing
Diagram: IOS Load Balancing. Source Cisco.

 

 

  • A key point: Video on IP Forwarding
The following video discusses the role of IP forwarding in networking. We will start by discussing switches and VLANs and then move to the basics of IP forwarding. So we have networks that are broken down into different VLANs. So will have a group of switches linked together via trunk ports that provide connectivity for VLANs across different physical distances.
IP forwarding, or packet forwarding, is routing data packets from one network interface to another based on their destination IP address. It enables data transfer between different networks, allowing communication between devices that are not directly connected.

 

IP Forwarding
Prev 1 of 1 Next
Prev 1 of 1 Next

 

Load Balancer Scalability

Scaling load balancers with Scale-Up or Scale-Out

a) Scale-up – Expand linear by buying more considerable servers, adding CPU and memory, etc. Scale-up is usually done on transaction database servers as these servers are difficult to scale out. Scaling up is a simple approach but the most expensive and nonlinear. Old applications were upgraded by scaling up ( vertical scaling )—a rigid approach that is not elastic. In a virtualized environment, applications are scaled linearly in a scale-out fashion.

b) Scale-out – Add more parallel servers, i.e., scaling linearly. Scaling out is easier on web servers; add additional web servers as needed. Netflix is an example of a company that designs by scale-out. It spins up Virtual Machines ( VM ) on-demand due to daily changes in network load. Scaling out is elastic and requires a load-balancing component. It is an agile approach to load balancing.

Shared states limit the scalability of scale-out architectures, so try to share and lock as few states as possible. An example of server locking is the eventual consistency approach used by Amazon, which limits the amount of transaction locking – shopping cards are not checked until you click “buy.”

 

  • Additional information: Scale up load balancing

A load balancer scale-up is the process of increasing the capacity of a load balancer by adding more computing resources. This can increase the system’s scalability or provide redundancy in case of system failure. The primary goal of scaling up a load balancer is to ensure the system can handle the increased workload without compromising performance.

Scaling up a load balancer involves adding more hardware and software resources, such as CPUs, RAM, and hard disks. These resources will enable the system to process requests more quickly and efficiently. When scaling up a load balancer, consider its architecture and the types of requests it will handle.

Different types of requests require different amounts of computing resources. For example, if the load balancer handles high-volume requests, it is essential to ensure that the system has enough CPUs and RAM to handle the requests.

Considering the network topology when scaling up a load balancer is also essential. The network topology defines how the load balancer will communicate with other systems, such as web servers and databases. If the network topology is not configured correctly, the system may be unable to handle the increased load.  Finally, monitoring the system after scaling up a load balancer is essential. This will ensure that the system performs as expected and that the increased capacity is used effectively. Monitoring the system can also help detect potential issues or performance bottlenecks.

By scaling up a load balancer, organizations can increase the scalability and redundancy of their system. However, it is essential to consider the architecture, types of requests, network topology, and monitoring when scaling up a load balancer. This will ensure the system can handle the increased workload without compromising performance.

 

  • Additional information: Scale-out load balancing

Scaling out a load balancer adds additional load balancers to distribute incoming requests across multiple nodes evenly. The process of scaling out a load balancer can be achieved in a variety of ways. Organizations can use virtualization or cloud-based solutions to add additional load balancers to their existing systems. Some organizations prefer to deploy their servers or use their existing hardware to scale the load balancer.

Regardless of the chosen method, the primary goal should be to create a reliable and efficient system that can handle increasing requests. This can be done by evenly distributing the load across multiple nodes, ensuring that every node is manageable and manageable. Additionally, organizations should consider deploying additional load balancer resources, such as memory, disk space, or CPU cores.

Finally, organizations should constantly monitor the load balancer’s performance to ensure the system runs optimally. This can be done by tracking the load-balancing performance, analyzing the response time of requests, and providing that the system can handle unexpected spikes in traffic.

 

Load Balancer Scalability: The Operations

The virtual IP address and load balancing control plane

Outside is a VIP, and inside, a pool of servers exists. A load balancer scaling device is configured for rules associating outside IP and port numbers with an inside pool of servers. Clients only know the outside IP address through, for example, DNS replies. The load-balancing control plane monitors the servers’ health and determines which can accept requests.

The client sends a TCP SYN packet, which the load balancer device intercepts. The load balancer carry’s out a load-balancing algorithm and sends it to the best-server destination. To get the request to the server, you can use Tunnelling, NAT, or two TCP sessions. In some cases, the load balancer will have to rewrite the content. Whatever the case, the load balancer has to create a session to know that this client is associated with a particular inside server.

 

Local and global load balancing

Local server selection occurs within the data center based on server load and application response times. Any application that uses TCP or UDP protocols can be load balanced. Whereas local load balancing determines the best device within a data center, global load balancing chooses the best data center to service client requests.

Global load balancing is supported through redirection based on  DNS and HTTP. HTTP mechanism provides better control, while DNS is fast and scalable. Both local and global appliances work hand-in-hand; the local device feeds information to the global device, enabling it to make better load-balancing decisions.

 

Load Balancer Scaling Types

Application-Level Load Balancer Scalability: Load balancing is implemented between tiers in the applications stack and carried out within the application. It is used in scenarios where applications are coded correctly, making it possible to configure load balancing in the application. Designers can use open-source tools with DNS or another method to track flows between tiers of the application stack.

Network-Level Load Balancer Scalability: Network-level load balancing includes DNS round-robin, Anycast, and Layer 4 – Layer 7 load balancers. Web browser clients do not usually have built-in application layer redundancy, which pushes designers to look at the network layer for load-balancing services. If applications were designed correctly, load balancing would not be a network-layer function.

 

 Application-level load balancing

The application-level load balancer scaling is about what we can do inside the application for load-balancing services. The first thing you can do is scale-up – add a more-worker process. Clients issue requests that block some significant worker processes and that resource is tied to TCP sessions. If your application requires session persistence ( long-lived TCP sessions ), you block worker processes even if the client is not sending data. The solution is FastCGI or changing the webserver to Nginx.

scaling load balancer

 

  • A key point: Nginx

Nginx is event-based. On Apache ( not event-based), every TCP connection consumes a worker process, but with Nginx, a client connection takes no processes unless you are processing an actual request. Generally, Linux is poor at processing many simultaneous requests.

Nginx does not use threads and can easily have 100,000 connections. With Apache, you lose 50% of the performance, and adding CPU doesn’t help. With around 80,000 connections, you will experience severe performance problems no matter how many CPUs you add. Nginx is by far a better solution if you expect a lot of simultaneous connections.

 

Example: Load Balancing with Auto Scaling groups on AWS.

The following looks at an example of load balancing in AWS. Registering your Auto Scaling group with an Elastic Load Balancing load balancer helps you set up a load-balanced application. Elastic Load Balancing works with Amazon EC2 Auto Scaling to distribute incoming traffic across your healthy Amazon EC2 instances.

This increases the scalability and availability of your application. In addition, you can enable Elastic Load Balancing within multiple Availability Zones to increase the fault tolerance of your applications. Elastic Load Balancing supports different types of load balancers. A recommended load balancer is the Application Load Balancer.

 

Elastic Load Balancing in the cloud.
Diagram: Elastic Load Balancing in the cloud. Source Amazon.

 

Network-based load balancing

First, try to solve the load balancer scaling in the application. When you cannot load balance solely using applications, turn to the network for load-balancing services. 

 

DNS round-robin load balancing

The most accessible type of network-level load balancing is DNS round robin. DNS server that keeps track of application server availability. The DNS control plane distributes user traffic over multiple servers round-robin. However, it does come with caveats:

  1. DNS does not know server health.
  2. DNS caching problems.
  3. No measures are available to prevent DoS attacks against servers.

Clients ask for the IP of the web server, and the DNS server replies with an IP address using some random order. It works well if the application uses DNS. However, some applications use hard-coded IP addresses; you can’t rely on DNS-based load balancing in these scenarios.

DNS load balancing also requires low TTL times, so the client will often ask the servers. Generally, DNS-based load balancing works well but not with web browsers. Why? DNS pinning.

 

DNS pinning

This is because there have been so many attacks on web browsers, and browsers now implement a security feature-called DNS pinning. DNS pinning is a method whereby you get the server’s IP address, and even though the TTL has expired, you ignore the DNS TTL and continue to use the URL.

It prevents people from spoofing DNS records and is usually built-in to browsers. DNS load balancing is perfect if the application uses DNS and listens to DNS TTL times. But unfortunately, web browsers are not in that category.

 

IP Anycast load balancing

IP Anycast provides geographic server load balancing. The idea is to use the same IP address on multiple POPs. Routing in the core will choose the closest POP, routing the client to the nearest POP. All servers have the same IP address configured on loopback.

Address Resolution Protocol (ARP) replies would clash if the same IP address were configured on the LAN interface. Use any routing mechanism to generate an Equal Cost Multi-Path (ECMP) for loopback addresses. For example, static routes are based on IP SLA, or you can use OSPF between the server and router.

 

Best for UDP traffic

As requests come in, the router will load balance based on 5-tuple. Do not load the balance on destination addresses /ports, as it’s always the same. It is usually done on the source client’s IP address/port number. The process takes the 5-tuple and creates a hash value, which creates independent paths based on that value. This works well for UDP traffic and how root servers work. Good for DNS server load balancing.

It works well for UDP as every request from the client is independent. TCP does not work like this, as TCP has sessions. It recommended not to use Anycast load balancing for TCP traffic. You need an actual load balancer if you want to load-balance TCP traffic. This could be a software package, Open Source ( HAproxy ), or a dedicated appliance.

 

Scaling load balancers at Layer 2

Layer 2 designs refer to the load balancer in bridged mode. As a result, all load-balanced and non-load-balanced traffic to and from the servers goes through the load-balancing device. The device bridges two VLANs together in the same IP subnet. Essentially, the load balancer acts as a crossover cable, merging two VLANs.

The critical point is that the client and server sides are in the same subnet. As a result, layer 2 implementations are much more accessible than layer 3 implementations, as there are no changes to IP addresses, netmasks, and default gateway settings on servers. But with a bridged design, be careful about introducing loops and implementing spanning tree protocol ( STP ).

 

Scaling load balancers at Layer 3 

With layer 3 designs, the load-balancing device acts in routed mode. Therefore, all load-balanced and non-load-balanced traffic to and from the server goes through the load-balancing device. The device routes between two different VLANs that are in two different subnets.

The critical point and significant difference between layer 3 and layer 2 designs are client-side VLANs and server-side VLANs in different subnets. Therefore, the VLANs are not merged, and the load-balancing device routes between VLANs. Layer 3 designs may be more complex to implement but will eventually be more scalable in the long run.

 

Scaling load balancers with One-ARM mode

One-armed mode refers to a load-balancing device, not in the forwarding path. The critical point is that the load balancer resides on its subnet and has no direct connectivity with server-side VLAN. A vital advantage of this model is that only load-balanced traffic goes through the device.

Server-initiated traffic bypasses the load balancer. Changes both source and destination IP address. The load balancer terminates outside TCP sessions and initiates new inside TCP sessions. When the client connection comes in, you take the source IP and port number, put them in connection tables, and associate it with the load balancer’s TCP port number and IP.

As everything comes from the load balance IP address, the servers can no longer see the original client. On the right-hand side of the diagram below, the source and destination traffic flow on the server side is the load balancer. The VIP addresses 10.0.0.1, and that is what the client connects to.

 

one arm mode load balancing

The use of X-forwarder-for HTTP header

To indicate to the server which the original client is, we use – X-forwarder-for HTTP header. The client’s IP address is replaced with the load balancer IP address. The load balancer can insert the X-Forwarders-for HTTP header where they copy the original IP address of the client into the extra HTTP header – “X-forward-for header.” Apache has a standard that copies the value of this header into the standard CGI variable so all the scripts can pretend no load balancer exists.

The load balancer inserts data into the TCP session; in other words, it has to take ownership of the TCP sessions, so it needs to take control of TCP activities, including buffering, fragmentation, and reassembling. Modifying HTTP requests is hard. F5 has an accelerated mode of TCP load balancing.

 

Scaling load balancers with Direct Server Return

Direct Server Return is when the same IP address is configured on all hosts. The same IP is configured on the loopback interface, not the LAN interface. The LAN IP address is only used for ARP, so the load balancer would send ARP requests only for the LAN IP address, rewrite the MAC header ( not TCP or HTTP alterations ), and send the unmodified IP packet to the selected server.

The server sends the reply to the client and does not involve the load balancer. Requires layer 2 connectivity between the load balancer and servers as load balancing is done on the MAC address ( example: Linux Virtual Server ). Also, a tunneling method that uses Layer 3 between the load balancer and servers is available.

Direct Server Return
Diagram: Direct Server Return.

 

  • A key point: MTU issues

If you do not have layer 2 connectivity, you can use tunnels, but be aware of MTU issues. Make sure the Maximum Segment Size ( MSS ) on the server is reduced so you do not have a PMTU issue between the client and server.

With direct server return, how do you ensure the reply is from the loopback, not the LAN address? If you are using TCP, the TCP session’s IP address is dictated by the original TCP SYN packet, so this is automatic.

However, UDP is different as UDP leaving is different from UDP coming in. So, in UDP cases, you need to set the IP address manually with the application or with iptables. But for TCP, the source in the reply is always copied from the destination IP address in the original TCP SYN request.

 

Scaling load balancers with Microsoft network load balancing

Microsoft load balancing is the ability to implement load balancing without load balancers. Instead, create a cluster IP address for the server and then use the flooding behavior to send it to all servers. 

Clients send a packet to the shared cluster IP address associated with a client’s MAC address. This cluster MAC does not exist anywhere. When the request arrives at the last Layer 3 switch, it sends an ARP request “Who has this IP address“?.

ARP request arrives at all the servers. So, when the client packet arrives, it sends to the bogus MAC address of the cluster, and because the MAC address has never been associated with any source, all the traffic is flooded from the Layer 2 switch to the servers. The performance of the Layer 2 switch falls massively as unicast flooding is done in software.

 

The use of Multicast

Microsoft then changed this to use Multicast. This does not work, and packets are dropped as an illegal source MAC to use a multicast MAC address. Cisco routers drop ARP packets with the source MAC address as multicast. Overcome this by configuring static ARP entries. Microsoft also implements IGMP to reduce flooding.

 

Load Balancing Options

User session persistence ( Stickiness )

The load balancer must keep all sessions state, even for inactive sessions. Session persistence creates much more state than just the connection table. Some web applications store client session data on the servers, so the sessions from the same client must go to the same server. This is particularly important when SSL is deployed for encryption or where shopping carts are used.

The client establishes an HTTP session with the webserver and logs in. After login, the HTTPS session from the same client should land on the same web server to which the client logged in using the initial HTTP request. The following are ways load balancers can determine who the source client is.

 

session persistance
Diagram: Scaling load balancers and session persistence.

 

  • Source IP address – > Problem may arise with large-scale NAT designs.
  • Extra HTTP cookies – > May require the load balancer to take ownership of the TCP session.
  • SSL session ID -> Will retain session persistence even if the client is roaming and the client’s IP address changes.

 

Data path programming

F5 uses scripts that act on packets triggering the load-balancing mechanism. You can select the server, manipulate HTTP headers or even manipulate content. For example, the load balancer can add caching headers in MediaWiki (which does not change content / caching headers ). The load balancer adds the headers that allow the content to be cached.

 

Persistent HTTP sessions

The client has a long-lived HTTP session to eliminate one RTT and congestion window problem; then, we have a short-lived session from the load balancer to the server. SPDY is a next-generation HTTP with multiple HTTP sessions over one TCP session. This is useful in high-latency environments such as mobile devices. F5 has a SPDY-to-HTTP gateway.

 

Destination-only NAT

Rewrites the destination IP address to a destination IP of the actual server and then forwards the packet. The reply packet has to hit the load balancer as the load balancer has to replace the server’s source IP with the load balancer’s source IP. The client IP does not change, so the server is talking directly with the client. This allows the server to do address-based access control or GEO location based on the source address.

 

Summary: How to scale load balancer

This post first addressed the different load balancer scalability options that consist of scale-up and then scale-out. The scale-out is generally the path of scaling load balancers we see today. It is less expensive and easier to perform. We then discussed how to scale the load balancer and the load balancer scalability options in the application and at a network level load balancing.

We also discussed the different design types of load balancing, such as user session persistence, destination-only NAT, and persistent HTTP sessions. There were several videos included that could provide more details on scaling load balancers.

So when you ask yourself how to scale load balancer, the first step is to examine the application. Can this be solved in the application, or do we need to push this to the network layer? Both have their pros and cons.

 

Types of Load Balancing:

There are various load-balancing techniques, each suited for different scenarios. Two common types are:

1. Round-robin: In this method, incoming requests are distributed sequentially across available servers, ensuring an even workload distribution. However, this technique does not consider the current server load, which might lead to uneven resource utilization.

2. Dynamic load balancing: This technique considers the current server load, distributing incoming requests based on server capacity, response time, or CPU utilization. This dynamically adjusting workload distribution method ensures optimal resource utilization and improved performance.

Conclusion:

In today’s technology-driven world, where downtime and performance degradation can have severe consequences, load balancing has become an essential component of modern computing. With its ability to enhance scalability, availability, and efficiency, load balancing ensures businesses can handle increased traffic, maintain high availability, and optimize resource utilization. By implementing load-balancing techniques, organizations can achieve a robust and reliable digital infrastructure, providing a seamless user experience and staying ahead in the competitive digital landscape.

 

 

load balancer scaling