SDX: Software Defined Internet Exchange

The source for this blog post is taken from Ivan Pepelnjak’s Software Gone Wild Podcast on Software Defined IXP and various publications from Laurent Vanbever. Laurent is an assistant professor at ETH Zürich and has come up with an interesting SDN and OSPF approach, which I wrote about here – Central Control OSPF SDN. The following post discusses his Software Defined IXP design, called SDX.


The majority of existing SDN solutions are aimed at cellular core networks, enterprises, and the data centre. At the WAN edge, SD-WAN is leading a solid path with many companies offering a BGP SDN solution augmenting natural Border Gateway Protocol (BGP) forwarding behavior with a controller architecture; optimizing both inbound and outbound Internet-bound traffic. How can we bring these existing SDN mechanisms to enhance BGP for Interdomain routing at Internet Exchange Points (IXP)?

IXPs are location points where networks from multiple providers meet to exchange traffic with BGP routing. Each participating AS exchanges BGP routes by peering eBGP with a BGP route server, which directs traffic to other network ASes over a shared Layer 2 fabric. The shared Layer 2 fabric provides the data plane forwarding of packets. The control plane used to exchange routing information is the actual BGP route server. A route server provides an alternative to full eBGP peering between participating AS members. It’s a control plane device and does not participate in data plane forwarding. There are currently around 300 IXPs worldwide. IXP are good locations to deploy SDN as their architecture is simple with flat networks. There is no routing for forwarding so there is a huge need for innovation. They usually consist of small teams making innovation easy to introduce. Fear is a one of the main emotions that prohibits innovation and one thing that creates fear is Loss of Service. This holds quite a large weight for IXP networks as they may have over 5 Terabytes of traffic per second. IXP are major connecting points and a slight outage can have a large ripple effect.

SDX, a software defined internet exchange, is an SDN solution originating from the combined efforts of Princeton and UC Berkeley. It aims to address IXP pain points (listed below) by deploying additional SDN controllers and OpenFlow enabled switches. It doesn’t try to replace the entire classical IXP architecture with something new, rather augments existing designs with a controller based solution, enhancing IXP traffic engineering capabilities. It is available as an open source software with a downloadable VM.


IXP Pain Points

BGP is great for scalability and reducing complexity but it severely limits how networks deliver traffic over the Internet. One very hard thing to do with BGP is proper inbound TE. The issue is that IP routing is destination based so the decision of where traffic enters the network is your neighbors. It’s not your decision. The forwarding mechanism is based on destination IP prefix. Meaning a device will forward all packets with the same destination address to the same next hop and the decision is made by the connected neighbor.


Main pain points for IXP networks:

Already mentioned, routing is based on destination IP prefix. BGP selects and exports routes for destination prefixes only. By default, it doesn’t match on other criteria in the packet header, such as source IP address or port number. It cannot application steer, which would be useful at IXP networks.

Secondly, you can only influence direct neighbors. There is no end-to-end control and it’s hard to influence neighbors that you are not peering with. Some BGP attributes don’t carry across multiple ASes and others may be recognized differently among different vendors. We also use a lot of de-aggregation to TE. Everyone is doing this, which is why we have the problem of 540,000 prefixes on the Internet. De-aggregation and multihoming create lots of scalability challenges.

Finally, there is an indirect expression of policy. Local Preference (LP) and Multiple Exit Discriminator (MED) are not great mechanisms to influence traffic engineering. We should have better inbound and outbound TE capabilities. MED, AS Path prepending and Local Preference are widely used attributes for TE but they are not the ultimate solution. They are inflexible in the sense that they can only influence routing decision based on destination prefixes. You can not do source IP or application type. They are very complex, involving intense configuration on multiple network devices.

All these solutions involve influencing the remote party to make a decision about how it enters your AS and if the remote party does not apply them correctly, TE becomes unpredictable.


SDX: Software Defined Internet Exchange

The SDX solution proposed by Laurent is a Software Defined Internet Exchange. Previously mentioned, it consists of a controller based architecture with OpenFlow 1.3 enabled physical switches. It aims to solve the pain points of BGP at the edge using SDN. SDN offers direct control over packet-processing rules that match on multiple header fields (not just destination prefix) and perform a variety of actions (not just forwarding); offering direct control over the data path. SDN enables the network to execute a far wider range of decisions concerning end-to-end traffic delivery.


How does it work?

The IXP fabric is replaced with OpenFlow-enabled switches. Now, TE is based on granular OpenFlow rules. It’s more predictable as it does not rely on 3rd party neighbors to make a decision about entry. OpenFlow rules can be based on any packet header field so it’s a lot more flexible than existing TE mechanisms. SDN-enabled data plane enables networks to have optimal WAN traffic with application steering capabilities.  

The existing route server does not get modified, but now we have the capability to push SDN rules into the fabric, without requiring classical BGP tricks (local preference, MED, AS prepend). The solution matches on destination MAC address, not destination IP prefix and uses ARP proxy to convert IP prefix to MAC addresses. The forwarding policies are defined by the participants and the role of the controller it to compile the forwarding entries into the fabric.

The SDX controller implementation has two main pipelines: a policy compiler, based on Pyretic; and a route server, based on ExaBGP. The policy compiler accepts input policies (custom route advertisements) written in Pyretic from individual participants as well as BGP routes from the route server. From this, it produces forwarding rules that implement the policies. The SDX controller combines the policies from multiple member ASes into one single policy for the physical switch implementation. The controller is like an optimized compiler; compiling down the policy and optimize the code in the forwarding by using a virtual next hop.

There are other potential design alternatives to SDX, for example, BGP FlowSpec. But in this case, BGP FlowSpec would have to be supported by all participating member AS edge devices.


About Matt Conran

Matt Conran has created 184 entries.

Leave a Reply