Data Center – Site Selection | Content routing
You can achieve redundancy and high availability by deploying multiple data centers to support distributed applications. The idea behind mutli data centers is that you can share load between them offering business continuity and data accessibility any time and anywhere.
Distributed workloads with multi-site architecture opens up a number of questions regarding the methods for site selection, path optimization for ingress / egress flows and data replication ( synchronous / asynchronous ) for storage. Once content is distributed to multiple data centers you need to manage the request for the distributed content and also manage the load by routing users requests for content to the appropriate data center. This is known as content routing. Content routing is the ability to take a users request and send it to the appropriate data center.
You can use different criteria to route users to the most optimum data centers. Proximity based site selection is selecting a data center that is geographically closer. Generally, this will improve response time. Additionally, you can route requests based on load of the data center or the application availability. Things start to become interesting when workloads want to move across geographically dispersed data center while maintaining active connections to front end users and backed systems. All these elements put increasing pressure on the data center interconnect ( DCI ) and the technology used to support workload mobility.
Site Selection – Multi-Site load distribution & Site-to-Site Recovery
Site selection can be used for site-to-site recovery and for multi-site load distribution. Multi-site load distribution requires mechanism that enable the same application to be accessed by both data centers i.e an active / active setup. For site-to-site load balancing, you must use an active / active scenario ( also known as hot standby ) in which both data centers host the same active application. Logically active / standby means that some application will be active on one site while those same applications will be standby at the other sites.
Data Center selection is key and determining which data center to target your request can be based on a number of factors such as proximity and load. Different applications will prefer different site selection mechanism. Video streaming will prefer the data center that is closest ( proximity selection ). Other types of applications would prefer data centers that are least loaded and others work efficiently with the standard round robin metric.
The three main traditional methods for Ingress site selection are DNS based, HTTP redirection and Route Health Injection.
Hypertext Transfer Protocol ( HTTP ) Redirection
Applications can have built-in HTTP redirection in their browsers. This enables them to communicate with a secondary server if the primary server is not available. When redirection is required the server will send an HTTP Redirect ( 307 ) to the client and sends the client to the correct site which has the required content. One advantage of this mechanism is that you have visibility into the content being requested but as you have probably already guessed it only works with HTTP traffic.
DNS based request routing
DNS based request routing point of control for geographic load distribution resides within DNS. DNS based request routing uses DNS for both site-to-site recovery and multi-site load distribution.
A DNS request either recursive or iterative is accepted from the client and that request is directed to a data center based on configurable parameters. This provides the ability to distribute the load among multiple data centers with an active / active design based on criteria such as least loaded, proximity, round robin and round trip time ( RTT ).
DNS based request routing becomes challenging if you have to support legacy applications that do not rely on DNS name resolution. These applications have hard-coded IP address that are used to communicate with other servers. When there is a combination of legacy and non-legacy applications, the solution might be to use both DNS based request routing in conjunction with IGP/BGP. Another caveat for this approach is that the rate of refresh for the DNS cache may impact the convergence time. There will also be increased traffic flow on the data center interconnect link once a VM moves to the secondary site – previously establish connections are hair-pinned.
Route Health Injection ( RHI )
RHI is implemented in front of the application and depending on its implementation allows the same address or different address to be advertised. Its basically a route injected by a local load balancer that is used to influence the ingress traffic path. RHI injects a static route when the VIP ( Virtual IP address ) becomes available and withdraws the static route when the VIP is no longer active. The VIP is used to represent an application.
Route Health Injection can be used for an active / active scenario as both data centers can use the same VIP to represent the server cluster for each application. RHI can create a lot of churn as routes are constantly being added and removed. If the number of applications supported grows, the number of host routes in the network grows linearly. The decision to use RHI should come down to scale and size of the data centers application footprint. RHI is commonly used on Intranets as the propagation of more specifics are not permitted on the Default Free Zone ( DFZ ). Certain requirements require RHI to be used in conjunction with BGP/IGP for external facing clients.
Due to the drawbacks of DNS caching, RHI is often preferred to DNS solution for Internet facing applications
BGP AS Prepending
This can be used for active / standby site selection and not an multi-load distribution method. BGP uses a best path algorithm to determine the best path to a specific destination. One of those steps that is widely used by all router manufactures is AS Path. The lower the number of AS’s in the path list, the better the route. Certain routes are advertised from both data centers with additional AS Paths added to the secondary sites routes. When BGP goes through its site selection processes, it will choose the path with the least amount of AS Paths i.e the primary site without AS Prepending.
BGP Conditional Advertisements
BGP Conditional Advertisements is useful when you are concerned that some manufactures may have AS Path explicitly removed. With conditional route advertisement, a certain condition must be met before an advertisement occurs. The routers on the secondary site monitor a set of prefix located on the first site and when those prefixes are not reachable at the first site the secondary sites begins to advertise. Its configuration is based on the use of community”no-export” and IBGP between the two sites. If routes were simply redistributed between BGP > IGP and advertised to the IBGP peer, the secondary site would simply advertise those routes defeating the purpose of a conditional advertisement.
The RHI method used either internally or externally with BGP is useful when you are forced to use IP as the site selection method. This may be the case when you have hard-coded IP addresses in the application used primarily with legacy applications or you are concerned about DNS caching issues. Site selection based on RHI and BGP requires no changes to DNS. However, its main drawback is that it cannot be used for active / active data centers and is primarily positioned as an active / standby method. This is because there is only ever one routing table entry in the routing table.
There are designs were you can use IP Anycast in conjunction with BGP, IGP and RHI to achieve an active / active scenario and I will discuss this in a later post. With this setup there is no need for BGP conditional route advertisement or AS Path prepending.