BGP Multipath Signaling
Border Gateway Protocol (BGP) was developed in 1989 to connect networks together and provide interdomain routing. The goal was to create a scalable non-chatty protocol. BGP grew in response to the overwhelming growth of the Internet and its use cases now vary from Multicast, DDoS protection, Layer 2 services and BGP Software Defined Networking (SDN). A lot of its success comes down to that fact that it is a very well known protocol. People know how to use BGP and additional features are easily added, making it very extensible and easy to use. It’s a lot easier to troubleshoot a BGP problem than a complex IGP problem. If you want to add something new you can create an attribute and simple traffic engineering can be done using predefined BGP communities. Many tools are available within the protocol. Recently, there has been infrastructure improvements such as keepalive and update generation enhancements, parallel route refresh, adaptive update cache size, and multipath signaling. This post will discuss the enhancements to BGP multipath signaling.
Best Path Only & Route Reflector Clusters
Multipathing enables BGP to send more than just the “best” path. It is useful in design where hot potato routing is broken. As soon as you install a route reflector (RR) you break hot potato routing and potentially create route oscillation. Route oscillations may occur in certain network topologies combined with certain MED configurations. To eliminate MED-induced route oscillations, a route reflector needs to advertise more than one path. A network that has a full mesh of iBGP speakers has consistent and equivalent routing information. It is free from MED-induced route oscillations and other routing inconsistencies. We need to find an approach where the RR advertise all the available paths for an address prefix or just the prefixes that may cause MED-induced route oscillations. As a general design best practice to achieve consistent routing, the IGP metrics for links within a route reflector cluster are smaller than the IGP metrics for the links between the route reflector clusters.
All transit providers want to protect the hot potato routing scheme for revenue reasons. Traffic consumes bandwidth and bandwidth cost money. Providers want traffic to leave their networks as soon as possible aka hot potato routing. The problem we have is when a route reflector receives two updates it only sends one. This is done by design for scalability reasons. BGP may also withdraw paths with lower policies (MED, Local Preference) resulting with only one NLRI announcement (diagram above). It was relevant for its time but for many reasons you might want to send more than one route. For example, faster convergence requires a primary and backup path. Another issue is that the route reflector selects the best path based on its own IGP and the route reflectors shortest exit point. Route reflector deployments will choose the egress router closest to the RR not for its individual clients. It selects the best path based on IGP metric computed from its IGP database and announces this path to clients. This is not optimum for egress traffic selection. As a result, traffic may travel longer paths to exit an AS. To combat this most service providers create a full mesh of route reflectors in all regions, resulting in a route reflector in every PoP. But if you have a large transit network an RR in every region comes at a considerable cost.
Multiple BGP Paths
There are a number of ways to get an RR or an ASBR to advertise more than one path 1) Different RD per prefix, 2) BGP Best External, 3) BGP Add-Path and 4) BGP Optimal Route Reflection (ORR).
Different RD (VPN identifier) per prefix is the recommended method for MPLS-VPN. If you are running Layer 3 VPN, you can assign different route distinguisher (RD) to the same prefix resulting in different IP address NLRI. Then the RR sees two different prefixes and will forward both. RR does the best path on two different VPNv4/v6 NLRI. With BGP Best External, you are telling the router not to withdraw an update even if it’s not the best one. It provides the network with an external backup route. The Add-path feature is a new BGP capability. It is an extension added to a BGP update where you can signal multiple paths to neighbors that must be negotiated at startup with all BGP neighbors. It’s the best method as long as you have good memory and if all nodes support it. All the information will be in the control plane and you can still do hot potato routing. There are many different types of add-path flavor including Add-n-path, Add-all-path and Add-all-multipath+backup.
BGP Optimal Route Reflection enables a virtual IGP location style design. It builds multiple RIB’s and computes the best path for each RIB. You need to influence your IGP to mimic what it would be like in other network locations. It essentially overwrites the default IGP location placement of the route reflector, enabling clients to direct traffic to their closest exit point in hot potato routing deployments.