Software Defined Internet Exchange

by Blog

Software Defined Internet Exchange

In today's digital era, where data is the lifeblood of every organization, the importance of a reliable and efficient internet connection cannot be overstated. As businesses increasingly rely on cloud-based applications and services, the demand for high-performance internet connectivity has skyrocketed. To meet this growing need, a revolutionary technology known as Software Defined Internet Exchange (SD-IX) has emerged as a game-changer in the networking world. In this blog post, we will delve into the concept of SD-IX, its benefits, and its potential to revolutionize how we connect to the internet.

Software Defined Internet Exchange, or SD-IX, allows organizations to dynamically connect to multiple Internet service providers (ISPs) through a centralized platform. Traditionally, internet traffic is exchanged through physical interconnections between ISPs, resulting in limited flexibility and control. SD-IX eliminates these limitations by virtualizing the interconnection process, enabling organizations to establish direct, secure, and scalable connections with multiple ISPs.

SD-IX Defined: Software Defined Internet Exchange, or SD-IX, is a cutting-edge technology that enables dynamic and automated interconnection between networks. Unlike traditional methods that rely on physical infrastructure, SD-IX leverages software-defined networking (SDN) principles to create virtualized interconnections, providing flexibility, scalability, and enhanced control.

Enhanced Performance: One of the prominent advantages of SD-IX is its ability to optimize network performance. By utilizing intelligent routing algorithms and traffic engineering techniques, SD-IX reduces latency, improves packet delivery, and enhances overall network efficiency. This translates into faster and more reliable connectivity for businesses and end-users alike.

Flexibility and Scalability: SD-IX offers unparalleled flexibility and scalability. With its virtualized nature, organizations can easily adjust their network connections, add or remove services, and scale their infrastructure as needed. This agility empowers businesses to adapt to changing demands, optimize their network resources, and accelerate their digital transformation initiatives.

Cost Efficiency: By leveraging SD-IX, organizations can significantly reduce their network costs. Traditional methods often require expensive physical interconnections and complex configurations. SD-IX eliminates the need for such costly infrastructure, replacing it with virtualized interconnections that can be provisioned and managed efficiently. This cost-saving aspect makes SD-IX an attractive option for businesses of all sizes.

Driving Innovation: SD-IX is poised to drive innovation in the networking landscape. Its ability to seamlessly connect disparate networks, whether cloud providers, content delivery networks, or internet service providers, opens up new possibilities for collaboration and integration. This interconnected ecosystem paves the way for novel services, improved user experiences, and accelerated digital innovation.

Enabling Edge Computing: As the demand for low-latency applications and services grows, SD-IX plays a crucial role in enabling edge computing. By bringing data centers closer to the edge, SD-IX reduces latency and enhances the performance of latency-sensitive applications. This empowers businesses to leverage emerging technologies like IoT, AI, and real-time analytics, unlocking new opportunities and use cases.

Software Defined Internet Exchange (SD-IX) represents a significant leap forward in the world of connectivity. With its virtualized interconnections, enhanced performance, flexibility, and cost efficiency, SD-IX is poised to reshape the networking landscape. As organizations strive to meet the ever-increasing demands of a digitally connected world, embracing SD-IX can unlock new realms of possibilities and propel them towards a future of seamless connectivity.

Matt Conran
SD-IX / SDX · INTERACTIVE LAB 01

How Does a Software-Defined Internet Exchange Change the Path?

Compare a conventional IXP fabric with a software-defined exchange. Follow the separation between BGP route exchange, the switching fabric and programmable traffic policy.

Internet Exchange Fabric READY
AS
Network A Member AS
IX
Exchange Fabric Shared switching fabric with BGP route exchange
BGP / Routing Control
AS
Network B Member AS
The exchange currently uses destination-based routing as the primary forwarding model.
Control Plane BGP
Data Plane L2 Fabric
Policy Granularity Prefix
Steering Limited
Conceptual model. Modern IXPs can use a variety of architectures, route servers, routing policies and programmable platforms. SDX is presented here as an architectural pattern rather than a claim that every IXP uses one specific controller or forwarding technology.
Exchange Result
Configure the exchange model and trace the traffic path.
Control-Plane Console
[READY] Awaiting exchange-path trace...
Engineering Insight

Traditional BGP-based exchange routing is powerful and scalable, but its normal decision model is centred on destination prefixes. A programmable exchange can introduce additional policy layers closer to the forwarding plane.

Highlights: Software Defined Internet Exchange

Understanding Software-Defined Internet Exchange

a) SD-IX is a cutting-edge technology that enables dynamic and flexible interconnection between networks. Unlike traditional internet exchange points (IXPs), SD-IX leverages software-defined networking (SDN) principles to create virtualized exchange environments. By abstracting the physical infrastructure, SD-IX allows on-demand network connections, enhanced scalability, and simplified network management.

b) Internet exchanges are physical locations where multiple Internet service providers (ISPs), content delivery networks (CDNs), and network operators connect their networks to exchange Internet traffic. By establishing direct connections, IXPs enable efficient and cost-effective data transfer between various networks, enhancing internet performance and reducing latency.

**How Internet Exchanges Work**

Internet Exchanges typically consist of high-speed switches and routers deployed in data centers. These devices provide the necessary connectivity between participating networks, facilitating traffic exchange.

To join an Internet Exchange, networks must adhere to specific peering policies and agreements. These guidelines dictate the terms of traffic exchange, including technical requirements, traffic ratios, and network security measures.

**Internet Exchange Points Around the World**

1: – ) Numerous Internet Exchange Points (IXPs) are located worldwide, with some of the most prominent ones including DE-CIX in Frankfurt, AMS-IX in Amsterdam, and LINX in London. These IXPs are critical hubs for global internet connectivity, enabling networks from different regions to exchange traffic.

2: – ) major global IXPs, regional and national Internet Exchange Points cater to specific geographic areas. These local IXPs further improve network performance by facilitating regional traffic exchange and reducing the need for long-haul data transfer.

3: – ) the demand for high-performance and reliable internet connectivity continues to grow, SD-IX is poised to play a pivotal role in shaping the future of networking. By virtualizing the interconnection process and providing organizations with unprecedented control and flexibility over their network connections, SD-IX empowers businesses to optimize their network performance, enhance security, and reduce costs. With its ability to scale on-demand and seamlessly reroute traffic, SD-IX is well-suited for the evolving needs of cloud-based applications, IoT devices, and emerging technologies such as edge computing.

4: – ) Defined Internet Exchange represents a paradigm shift in how organizations connect to the Internet. By virtualizing the interconnection process and providing enhanced performance, reliability, cost efficiency, scalability, and security, SD-IX offers a compelling solution for businesses seeking to optimize their network infrastructure. As the digital landscape continues to evolve, SD-IX is set to revolutionize the way we connect to the internet, enabling organizations to stay ahead of the curve and unlock new possibilities in the digital era.

Key SD-IX Considerations:

– Enhanced Performance and Latency Reduction: SD-IX brings networks closer to end-users by establishing globally distributed points of presence (PoPs). This proximity reduces latency and improves application performance, resulting in a superior user experience.

– Seamless Network Scalability: With SD-IX, organizations can quickly scale their network resources up or down based on demand. This agility empowers businesses to adapt rapidly to changing network requirements, ensuring optimal performance and cost-efficiency.

– Simplified Network Management: Traditional IXPs often require complex physical infrastructure and manual configurations. SD-IX simplifies network management by providing a centralized control plane, allowing administrators to automate provisioning, traffic engineering, and policy enforcement.

– Cloud Service Providers: SD-IX enables providers to establish direct and secure customer connections. This direct access bypasses the public internet, ensuring better security, lower latency, and improved data transfer speeds.

– Content Delivery Networks (CDNs): CDNs can leverage SD-IX to optimize content delivery by strategically placing their PoPs closer to end-users. This reduces latency, minimizes bandwidth costs, and enhances content delivery performance.

– Enterprises and Multi-Cloud Connectivity: Enterprises can benefit from SD-IX by establishing private connections between their networks and multiple cloud service providers. This enables secure, high-performance multi-cloud connectivity, facilitating seamless data transfer and workload migration.

Understanding SD-IX

At its core, SD-IX is an architectural framework enabling the dynamic and automated internet traffic exchange between networks. Unlike traditional methods that rely on physical infrastructure, SD-IX leverages software-defined networking (SDN) principles to create a virtualized exchange ecosystem. By decoupling the control plane from the data plane, SD-IX brings flexibility, agility, and scalability to internet exchange.

One of SD-IX’s critical advantages is its ability to provide enhanced performance through optimized routing. By leveraging intelligent algorithms and real-time analytics, SD-IX can intelligently direct traffic along the most efficient paths, reducing latency and improving overall network performance. Moreover, SD-IX offers improved scalability, allowing networks to dynamically adjust their capacity based on demand, ensuring seamless connectivity even during peak usage.

Security and Privacy Advancements

SD-IX brings significant advancements in an era where data security and privacy are of the utmost concern. With the ability to implement granular access control policies and encryption mechanisms, SD-IX ensures secure data transmission across networks. SD-IX’s centralized management and monitoring capabilities enable network administrators to detect and mitigate potential security threats in real-time, bolstering overall network security.

Software-defined networks

A software-defined network (SDN) optimizes and simplifies network operations by closely tying applications and network services, whether real or virtual. By establishing a logically centralized network control point (typically an SDN controller), the control point orchestrates, mediates, and facilitates communication between applications that wish to interact with network elements and network elements that want to communicate information with those applications. The controller exposes and abstracts network functions and operations through modern, application-friendly, bidirectional programmatic interfaces.

As a result, software-defined, software-driven, and programmable networks have a rich and complex history and various challenges and solutions to those challenges. Because of the success of technologies that preceded them, software-defined, software-driven, and programmable networks are now possible.IP, BGP, MPLS, and Ethernet are the fundamental elements of most networks worldwide.

Control and Data Plane Separation

SDN’s early proponents advocated separating a network device’s control and data planes as a potential advantage. Network operators benefit from this separation regarding centralized or semi-centralized programmatic control. As well as being economically advantageous, it can consolidate into a few places, usually a complex piece of software to configure and control, onto less expensive, so-called commodity hardware.

One of SDN’s most controversial tenets is separating control and data planes. It’s not a new concept, but the contemporary way of thinking puts a twist on it: how far should the control plane be from the data plane, how many instances are needed for resiliency and high availability, and if 100% of the control plane can be moved beyond a few inches are all intensely debated. There are many possible control planes, ranging from the simplest, the fully distributed, to the semi- and logically centralized, to the strictly centralized.

OpenFlow Matching

With OpenFlow, the forwarding path is determined more precisely (matching fields in the packet) than traditional routing protocols because the tables OpenFlow supports more than just the destination address. Using the source address to determine the next routing hop is similar to the granularity offered by PBR.

In the same way that OpenFlow would do many years later, PBR permits network administrators to forward traffic based on “nontraditional” attributes, such as the source address of a packet. However, PBR-forwarded traffic took quite some time for network vendors to offer equivalent performance, and the final result was very vendor-specific.

Example Technology: Policy Based Routing

**How Policy-Based Routing Works**

At its core, policy-based routing operates by applying a series of rules to incoming packets. These rules, defined by network administrators, determine the next hop for packets based on criteria such as source or destination IP address, protocol type, or even application-level data. Unlike conventional routing protocols that rely solely on destination IP addresses to make decisions, PBR provides the ability to consider a broader set of parameters, thus enabling more granular control over network traffic flows.

**Benefits of Implementing Policy-Based Routing**

One of the primary advantages of PBR is its ability to optimize network performance. By directing traffic along paths that make the most sense for specific types of data, network operators can reduce congestion and improve response times. Additionally, PBR can enhance security by allowing sensitive data to be routed over secure, encrypted pathways while less critical data takes a different route. This capability is particularly valuable in environments where network resources are shared across multiple departments or where specific compliance requirements must be met.

**Challenges and Considerations**

Despite its benefits, policy-based routing is not without challenges. The complexity of configuring and maintaining PBR rules can be daunting, especially in large networks with diverse requirements. Careful planning and ongoing management are essential to ensure that PBR implementations remain effective and do not introduce unintended routing behaviors. Moreover, network administrators must keep an eye on the broader network architecture to ensure that PBR policies align with overall network goals and do not conflict with other routing protocols in use.

**Use Cases: Real-World Applications of Policy-Based Routing**

Policy-based routing finds its place in a variety of real-world applications. In enterprise networks, PBR is often used to prioritize business-critical applications or to implement cost-saving measures by routing traffic over less expensive links when possible. It also plays a significant role in multi-tenant environments, where different customers or departments may require distinct levels of service. Additionally, PBR is instrumental in hybrid cloud environments, where data flows between on-premises infrastructure and cloud services must be managed efficiently.

**The Role of SDN Solutions**

Most existing SDN solutions are aimed at cellular core networks, enterprises, and the data center. However, at the WAN edge, SD-WAN and WAN SDN are leading a solid path, with many companies offering a BGP SDN solution augmenting natural Border Gateway Protocol (BGP) IP forwarding behavior with a controller architecture, optimizing both inbound and outbound Internet-bound traffic. So, how can we use these existing SDN mechanisms to enhance BGP for interdomain routing at Internet Exchange Points (IXP)?

**The Role of IXPs**

IXPs are location points where networks from multiple providers meet to exchange traffic with BGP routing. Each participating AS exchanges BGP routes by peering eBGP with a BGP route server, which directs traffic to another network ASes over a shared Layer 2 fabric. The shared Layer 2 fabric provides the data plane forwarding of packets. The actual BGP route server is the control plane to exchange routing information.

  1.  
SD-IX / SDX · INTERACTIVE LAB 02

What Can BGP Control — and What Needs a Programmable Policy?

Experiment with different traffic-engineering objectives and control mechanisms. See how destination-prefix routing differs from policies that can consider additional packet attributes.

Policy Compilation Path READY
PKT
Traffic Flow Destination prefix
POL
Policy Engine BGP route policy
FAB
IXP Fabric Forwarding decision
Available Match Fields Destination
Destination
Source
Protocol
Port
Application
Select a traffic objective and policy mechanism, then evaluate the design.
Policy Match Prefix
Path Choice Not Evaluated
Granularity Destination
Control Fit —
BGP Excellent for scalable inter-domain reachability and destination-prefix policy.
PBR Can introduce additional match criteria, but deployment and scale depend heavily on the network design.
SDX Adds a programmable policy layer capable of compiling richer traffic rules into the exchange fabric.
Conceptual simulation. The exact match fields, controller model, route-server behaviour and forwarding capabilities vary between implementations. This lab illustrates the architectural distinction rather than a specific SDX product.
Policy Evaluation
Awaiting evaluation...
Control-Plane Console
[READY] Policy engine awaiting input...
Engineering Insight

BGP is intentionally powerful at what it does: exchanging reachability information and selecting paths toward destination prefixes. More granular traffic steering requires additional mechanisms.

Software Defined Internet Exchange

An Internet exchange point (IXP) is a physical location through which Internet infrastructure companies such as Internet Service Providers (ISPs) and CDNs connect. These locations exist on the “edge” of different networks and allow network providers to share transit outside their network.

IXPs will run BGP.  Also, it is essential to understand that Internet exchange point participants often require that the BGP NEXT_HOP specified in UPDATE messages be that of the peer’s IP address, as a matter of policy.

Route Server

A route server provides an alternative to full eBGP peering between participating AS members, enabling network traffic engineering. It’s a control plane device and does not participate in data plane forwarding. There are currently around 300 IXPs worldwide. Because of their simple architecture and flat networks, IXPs are good locations to deploy SDN.

There is no routing for forwarding, so there is a huge need for innovation. They usually consist of small teams, making innovation easy to introduce. Fear is one of the primary emotions that prohibit innovation, and one thing that creates fear is Loss of Service.

This is significant for IXP networks, as they may have over 5 Terabytes of traffic per second. IXPs are major connecting points, and a slight outage can have a significant ripple effect.

  • A key point. Internet Exchange Design

SDX, a software-defined internet exchange, is an SDN solution based on the combined efforts of Princeton and UC Berkeley. It aims to address IXP pain points (listed below) by deploying additional SDN controllers and OpenFlow-enabled switches. It doesn’t try to replace the entire classical IXP architecture with something new but rather augments existing designs with a controller-based solution, enhancing IXP traffic engineering capabilities. However, the risks associated with open-source dependencies shouldn’t be ignored.

Challenges: Software Defined Internet Exchange: IXP Pain Points

BGP is great for scalability and reducing complexity but severely limits how networks deliver traffic over the Internet. One tricky thing to do with BGP is good inbound TE. The issue is that IP routing is destination-based, so your neighbor decides where traffic enters the network. It’s not your decision.

The forwarding mechanism is based on the destination IP prefix. A device forwards all packets with the same destination address to the same next hop, and the connected neighbor decides.

The main pain points for IXP networks:

As already mentioned, routing is based on the destination IP prefix. BGP selects and exports routes for destination prefixes only. It doesn’t match other criteria in the packet header, such as source IP address or port number. Therefore, it cannot help with application steering, which would be helpful in IXP networks.

Secondly, you can only influence direct neighbors. There is no end-to-end control, and it’s hard to influence neighbors that you are not peering. Some BGP attributes don’t carry across multiple ASes; others may be recognized differently among vendors. We also use a lot of de-aggregation to TE. Everyone is doing this, which is why we have the problem of 540,000 prefixes on the Internet. De-aggregation and multihoming create lots of scalability challenges.

Finally, there is an indirect expression of policy. Local Preference (LP) and Multiple Exit Discriminator (MED) are ineffective mechanisms influencing traffic engineering. We should have better inbound and outbound TE capabilities. MED, AS Path, pretending, and Local Preference are widely used attributes for TE, but they are not the ultimate solution.

They are inflexible because they can only influence routing decisions based on destination prefixes. You can not do source IP or application type. They are very complex, involving intense configuration on multiple network devices. All these solutions involve influencing the remote party to decide how it enters your AS, and if the remote party does not apply them correctly, TE becomes unpredictable.

SDX: Software-Defined Internet Exchange

The SDX solution proposed by Laurent is a Software-Defined Internet Exchange. As previously mentioned, it consists of a controller-based architecture with OpenFlow 1.3-enabled physical switches. It aims to solve the pain points of BGP at the edge using SDN.

Transport SDN offers direct control over packet-processing rules that match on multiple header fields (not just destination prefixes) and perform various actions (not just forwarding), offering direct control over the data path. SDN enables the network to execute a broader range of decisions concerning end-to-end traffic delivery.

How does it work?

What is OpenFlow? Is the IXP fabric replaced with OpenFlow-enabled switches? Now, network traffic engineering is based on granular OpenFlow rules. It’s more predictable as it does not rely on third-party neighbors to decide the entry. OpenFlow rules can be based on any packet header field, so they’re much more flexible than existing TE mechanisms. An SDN-enabled data plane enables networks to have optimal WAN traffic with application steering capabilities. 

The existing route server has not been modified, but now we can push SDN rules into the fabric without requiring classical BGP tricks (local preference, MED, AS prepend). The solution matches the destination MAC address, not the destination IP prefix, and uses an ARP proxy to convert the IP prefixes to MAC addresses.

The participants define the forwarding policies, and the controller’s role is to compile the forwarding entries into the fabric. The SDX controller implementation has two main pipelines: a policy compiler based on Pyretic and a route server based on ExaBGP. The policy compiler accepts input policies (custom route advertisements) written in Pyretic from individual participants and BGP routes from the route server. This produces forwarding rules that implement the policies.

SDX Controller

The SDX controller combines the policies from multiple member ASes into one policy for the physical switch implementation. The controller is like an optimized compiler, compiling down the policy and optimizing the code in the forwarding by using a virtual next hop. There are other potential design alternatives to SDX, such as BGP FlowSpec. But in this case, BGP FlowSpec would have to be supported by all participating member AS edge devices.

Closing Points on Software Defined Internet Exchange

At its core, SDX is an evolution of traditional Internet Exchange Points (IXPs), which are critical nodes in the internet’s infrastructure, allowing different networks to interconnect. Traditional IXPs are hardware-driven, requiring physical switches and routers to manage traffic between networks. SDX, on the other hand, leverages the principles of SDN to introduce a software layer that enhances flexibility and control over these exchanges. This software-defined approach allows for dynamic configuration and management of network policies, enabling more efficient and tailored data traffic handling.

One of the primary benefits of SDX is its capacity for greater agility and adaptability in managing network traffic. Unlike traditional IXPs, SDX can quickly respond to changing network demands, optimizing the flow of data in real time. This adaptability is particularly beneficial for handling peak traffic periods or unexpected surges, ensuring that data exchanges remain smooth and uninterrupted. Additionally, SDX provides enhanced security features, as the software layer can be programmed to detect and mitigate potential threats more effectively than conventional hardware solutions.*

The implications of adopting SDX are vast and varied. For internet service providers, SDX offers the potential to provide more personalized services to their customers, adjusting bandwidth and routing protocols based on individual needs. Enterprises can benefit from SDX by gaining more control over their data exchanges, optimizing their network performance, and reducing operational costs. Furthermore, SDX is particularly advantageous for emerging technologies like the Internet of Things (IoT) and 5G networks, where the ability to efficiently handle large volumes of data is crucial.

Despite its many advantages, the transition to SDX is not without its challenges. Implementing SDX requires significant changes to existing network infrastructures, which can be costly and complex. Moreover, the shift to a software-centric model necessitates a new skill set for IT professionals, who must be adept in both networking and software development. There is also the consideration of interoperability, as networks must ensure that their SDX solutions can work seamlessly with other networks and legacy systems.

SD-IX / SDX · INCIDENT CHALLENGE

SDX Traffic Engineering Challenge

A customer wants only one class of traffic to use a different exchange path. Can conventional BGP policy solve the requirement, or does the problem need a more granular traffic-steering layer?

!
Incident: Video Traffic Is Taking the Wrong Exit AS65010 has two exchange paths. Peer A is congested, while Peer B has available capacity. General routing remains healthy, but the customer wants only video traffic to prefer Peer B.
AS65010 Customer Network
→
Peer A Congested · 92%
or
Peer B Healthy · 38%
→
Destination Content Network
BGP Sessions Established
Reachability Healthy
Peer A Congested
Peer B Available
Requirement Video Only
What is the best engineering response?
Engineering principle: when the requirement is “change the path for this particular class of traffic while leaving other traffic unchanged,” the key question is not simply whether another BGP path exists. The key question is whether the chosen control plane can express the required classification.
Challenge Score
Result — / 4
Select the response you believe best matches the evidence.
Incident Console
[READY] Incident loaded.
[BGP] Peer sessions established.
[PATH] Multiple exchange paths available.
[ALERT] Peer A congestion detected.
Engineering Insight

The important clue is “video only.” A global routing preference may affect more traffic than intended. The engineering challenge is therefore one of policy granularity, not basic reachability.

Routing Control Platform

BGP-based Routing Control Platform (RCP)

by Blog

Routing Control Platforms: Centralised Intelligence for BGP

How can a network make routing decisions using a wider view of topology, policy and reachability than any individual router has locally? A Routing Control Platform (RCP) explores one answer: separate the computation of routing decisions from the routers that forward packets.

In a traditional BGP design, routers select paths using the routing information and policies available to them. Route Reflectors improve iBGP scalability, but their path-selection perspective can affect which routes are advertised to clients. An RCP-style architecture combines BGP information with an IGP topology view to compute routing decisions with a broader understanding of the network.

The engineering question: Can centralised routing intelligence improve path selection and scalability without placing the controller in the packet-forwarding path—and what happens when its network view is incomplete or stale?
01 · SEE

Understand the architecture

Explore how IGP topology, BGP reachability and routing policy feed a control platform that computes route decisions.

02 · OBSERVE

Compare routing perspectives

Examine why a Route Reflector and a controller using per-router topology information may choose different exits.

03 · INVESTIGATE

Diagnose incorrect decisions

Correlate topology freshness, IGP costs, BGP state and controller output to identify the cause of a suboptimal route.

Engineering outcome: understand the distinction between centralised route computation and distributed packet forwarding, evaluate RCP against Route Reflector designs, and identify the evidence needed to validate routing correctness.
NETWORK INSIGHT · ENGINEERING GUIDE
Routing Control Platform — Investigation Path

Follow the routing decision from architecture to operational verification. Each stage builds on the evidence collected in the previous one.

ENGINEERING ORIENTATION · 01 / SEE

How a routing decision travels through RCP

Select each stage to follow the control-plane workflow, from network information to packet forwarding.

Discover the networkThe platform needs a view of IGP topology, BGP routes and relevant policy before it can calculate a useful routing decision.
Section 1 · See the architecture

RCP Architecture: From Network State to Routing Decision

A Routing Control Platform (RCP) is an architectural approach to computing BGP routing decisions with a broader view of network topology, reachability and policy. Rather than relying exclusively on each router's local view, an RCP-style system collects routing information, evaluates it centrally and communicates routing decisions back to routers.

The important distinction is between where a routing decision is calculated and where packets are forwarded. RCP moves part of the route-computation process into a control platform. The routers still perform the actual packet forwarding using their local forwarding information base (FIB).

The engineering question

Can a controller use BGP information and IGP topology to make better-informed routing decisions without becoming a dependency in the normal packet-forwarding path?

1.1 The components of an RCP-style architecture

The original RCP concept separates information collection, route computation and route distribution. Exact component names and implementation details vary, but the following model captures the core responsibilities.

INPUT · TOPOLOGY
IGP topology view

Collects information about the internal network, including links, reachability and routing metrics. This helps the controller estimate the internal cost from a relevant router to a BGP next hop or external exit.

INPUT · REACHABILITY
BGP information

Provides reachable prefixes and path attributes such as AS_PATH, LOCAL_PREF, MED and next-hop information. The available paths depend on BGP propagation, policy and the architecture used to expose routes to the platform.

COMPUTE · CONTROL
Route Control Server (RCS)

Combines the available routing information with topology and policy to calculate routing decisions. Depending on the design, it may evaluate decisions from the perspective of particular routers or regions.

OUTPUT · ROUTING
Router RIB and FIB

Routers receive and process routing information, install eligible routes and program their local forwarding tables. Their forwarding hardware or software then sends packets toward the selected next hop.

1.2 Follow the route-decision flow

A useful way to understand the design is to trace one destination prefix from discovery to forwarding. The platform needs enough accurate information to make a decision, and the routers need a valid route and resolvable next hop to forward traffic successfully.

CONTROL-PLANE WORKFLOW
01 · Discover Learn IGP topology and BGP reachability.
02 · Correlate Combine paths, metrics and routing policy.
03 · Calculate Evaluate the appropriate route decision.
04 · Distribute Routers process routes and program forwarding state.
Simplified conceptual flow: actual RCP implementations differ in how they learn routes, compute decisions and distribute results.

1.3 Control plane versus data plane

RCP is not the same as a controller that forwards every packet centrally. It is primarily concerned with the computation and distribution of routing information. Once a router has installed a route and resolved its next hop, packets are normally forwarded locally.

Function Control plane Data plane
Primary purpose Learn reachability and determine usable routes. Forward packets using installed forwarding entries.
RCP's role Correlate BGP information, topology and policy to calculate routing decisions. Not normally in the packet-by-packet forwarding path.
Router's role Maintain routing protocols and process received routing information. Resolve the destination against the FIB and forward toward the next hop.
Evidence to inspect BGP routes, attributes, IGP topology, policy and controller state. Installed route, FIB entry, next-hop resolution and observed traffic.

1.4 Why the topology view matters

BGP path selection and internal path costs answer different questions. BGP attributes and policy influence which routes are eligible and preferred; IGP information helps establish internal reachability and costs to next hops. An RCP design attempts to use both kinds of information when calculating a decision.

Consider two external exits advertising the same destination. A central platform might know that Exit A is closer to one region while Exit B is closer to another. That information can support more appropriate router-specific decisions, provided the architecture can calculate and distribute those decisions correctly.

This does not mean that the lowest IGP cost always wins. LOCAL_PREF, AS_PATH, MED, next-hop reachability, policy and the implementation's decision process can affect the outcome. Engineers must inspect the full decision chain rather than infer the chosen route from one metric.

1.5 The design's dependency: accurate, current state

Centralised computation creates a strong dependency on the quality and freshness of the information being used. If a link fails but the controller still sees an older topology, it may calculate a route using a path that is no longer available. The resulting problem may appear as a route-selection issue even when the underlying cause is stale state or failed next-hop reachability.

  • Topology: Does the controller's view match the current IGP state?
  • Reachability: Are the BGP route and its next hop still valid?
  • Policy: Are route attributes and policy applied as intended?
  • Distribution: Did the target router receive and accept the intended route?
  • Forwarding: Does the local FIB reflect the route, and does traffic succeed?

1.6 RCP and Route Reflectors are not interchangeable terms

Route Reflectors address the scaling problem of a full iBGP mesh by reducing the number of required iBGP sessions. However, a reflector generally advertises selected paths according to its BGP decision process and configuration. Clients may therefore not receive every available path, and the reflector's routing perspective may differ from a client's perspective.

RCP was proposed as an alternative architectural approach to route computation and distribution, aiming to combine BGP information with topology knowledge. It should not be treated as a universal replacement for Route Reflectors or as a guarantee of optimal routing. Results depend on the implementation, information available, policies and network design.

Section 1 engineering takeaway

RCP separates routing-decision computation from packet forwarding. To understand a route, trace the complete chain: topology and BGP inputs → policy-aware calculation → route distribution → router RIB/FIB → verified traffic outcome. The next section compares the routing perspectives behind RCP and Route Reflectors.

How Does a Routing Control Platform Make a Routing Decision?

An RCP does not sit in the packet-forwarding path. It builds a broader view of routing and topology, computes policy-aware decisions, and communicates those decisions back to the routers.

Control Plane
Network State
IG
IGP / Topology
Link-state information gives the controller visibility of network connectivity and costs.
→
Routing Inputs
BG
BGP Information
The platform learns reachable prefixes, attributes and policy-relevant routing information.
→
Control Platform
R
Route Control Server
Combines topology, BGP state and policy to calculate the appropriate route for each perspective.
→
Decision Output
IB
Routing Update
The resulting route decision is distributed to the relevant BGP speakers.
→
Forwarding
F
Router FIB
Routers continue to forward packets locally using the route installed in their forwarding plane.
Topology Visibility
The controller needs a complete network view.

The key RCP idea is visibility. Instead of making a routing decision from the limited perspective of one router, the platform combines IGP topology information with BGP reachability and policy information.

Data Plane Routers forward traffic locally
Control Plane RCP computes routing decisions
Design Goal Global visibility + scalable distribution
rcp> Collecting IGP topology + BGP reachability → building routing view...
ENGINEERING ORIENTATION · 02 / OBSERVE

Who has the right view of the network?

Choose a routing model to examine how its decision-making perspective can affect exit selection.

Available evidenceThe reflector learns BGP paths it receives and evaluates them using its own routing information and policy context.
Engineering questionDoes the selected path suit the forwarding router, or only the reflector's perspective?
What to verify: Compare the selected BGP path with the client router's IGP cost, next-hop reachability and local policy. A reflector's best path is not automatically the best path for every client.
Section 2 · Observe routing perspectives

Compare Route Reflector and RCP Perspectives

Two routers can reach the same destination but have different internal costs to the available exits. Understanding which routing information is visible—and whose perspective drives the decision—is essential when investigating unexpected BGP paths.

Route Reflectors improve iBGP scalability by reducing the need for a full mesh of internal BGP sessions. A Routing Control Platform takes a different architectural approach: it seeks to combine BGP reachability and policy with a wider view of the IGP topology when calculating routing decisions.

Observation is not the same as optimisation

A controller or reflector can only make a decision from the information and policies available to it. To establish whether a route is appropriate, compare the selected BGP path, the relevant router's IGP costs, next-hop reachability and the actual forwarding state.

2.1 What does a Route Reflector see?

In a conventional route-reflector design, clients advertise routes to the reflector, which applies BGP selection and reflection rules. The reflector generally advertises selected paths rather than every path it learns. This reduces control-plane overhead, but it can limit which alternatives are visible to clients.

SCALABILITY
Fewer iBGP sessions

Route reflection reduces the need for every internal BGP router to peer directly with every other router. This simplifies session management as the network grows.

PATH VISIBILITY
Selected paths are reflected

Standard route reflection can hide alternatives that were not selected by the reflector. Additional mechanisms or design choices may be needed when greater path diversity is required.

ROUTING PERSPECTIVE
The reflector has its own view

The reflector's BGP decision is made in its own routing context. Its IGP cost to an exit can differ from the cost seen by a client in another part of the network.

VALIDATION
Check the receiving router

The reflected route must still be evaluated in the receiving router's context, including its installed next hop, local forwarding state and reachability.

2.2 What does an RCP-style design add?

RCP was proposed to address limitations associated with distributed route computation and route reflection. By combining BGP information with an IGP topology view, a controller can attempt to calculate decisions that account for the internal cost from a particular router or region to an exit.

The goal is not simply to select one globally best exit. In some designs, the appropriate choice differs by router because internal topology and policy differ. A platform that models these differences may be able to make better-informed decisions than one relying solely on a single reflector's perspective.

Example · Two exits, two router perspectives
Exit A IGP cost from West: 20
IGP cost from East: 70
Exit B IGP cost from West: 40
IGP cost from East: 15

Illustrative IGP costs only. If the relevant BGP paths and policy permit these choices, West may favour Exit A while East may favour Exit B. Actual BGP selection is not determined by IGP cost alone.

2.3 Compare the models using evidence

Engineering question Route Reflector RCP-style approach
What is the routing input? Received BGP routes, path attributes, local topology and configured policy. BGP reachability and policy combined with a collected view of IGP topology.
How are decisions made? The reflector applies its BGP decision process to the paths it knows. The platform calculates decisions using its available routing information and design-specific logic.
Can path diversity be limited? Yes. Conventional reflection may advertise only selected paths. Potentially different path visibility, depending on how the platform learns and distributes routes.
Can topology freshness matter? Yes. Local IGP state and next-hop reachability affect routing outcomes. Yes. Stale or incomplete topology information can undermine a calculated decision.
What proves the outcome? Received route, BGP attributes, local route selection and FIB. Input freshness, computed decision, distributed route, router RIB/FIB and traffic results.

2.4 Why the shortest internal path may not win

A common troubleshooting mistake is to assume that the router must use the exit with the lowest IGP cost. BGP selection is policy-driven and considers several attributes and decision steps. LOCAL_PREF, AS_PATH, MED, next-hop validity and implementation-specific configuration can influence which route is selected before or alongside internal forwarding considerations.

For example, a higher LOCAL_PREF can cause one path to be preferred even when its exit is farther away in the IGP. That can be intentional: an organisation may prefer a particular transit provider for commercial, security or traffic-engineering reasons. The engineer must first determine the intended policy, then compare the observed decision against it.

2.5 Build a reliable observation checklist

  • Route availability: Does the relevant router know the destination prefix?
  • Path attributes: Which candidate routes are visible, and what are their LOCAL_PREF, AS_PATH and MED values?
  • Router perspective: What are the IGP costs and next-hop reachability from the affected router?
  • Reflection or distribution: Was the intended route actually advertised to the client?
  • Controller freshness: If RCP is involved, does its topology snapshot reflect the current network?
  • Forwarding state: Does the local FIB match the expected route, and can traffic reach the destination?
Section 2 engineering takeaway

Do not judge a routing architecture by the route it selects in isolation. Compare the paths it can see, the topology and policy it uses, the router for which the decision is intended, and the resulting forwarding state. In the interactive lab that follows, compare exit selection from different router perspectives before investigating a deliberately incorrect decision in Section 3.

RCP Incident: Why Did One Region Get the Wrong Exit?

A routing incident has been reported. Reachability is healthy, BGP sessions are established, but one region is taking a suboptimal exit. Examine the evidence and identify the real control-plane problem.

Troubleshooting
Incident Report

Traffic from the West region is exiting through Exit B even though Exit A is significantly closer from the West router's IGP perspective. No BGP session is down.

BGP Session Established
Prefix Reachability Available
West → Exit A 18 IGP cost
West → Exit B 61 IGP cost
RR Selected Path Exit B
Controller Topology Stale by 9 min

What is the most likely root cause?

Network State West is closer to Exit A
→
Control Decision RR perspective selects Exit B
→
Topology Visibility Controller information is stale
→
Corrective Action Refresh topology and compute per-router paths
Engineering Insight

RCP does not simply mean “put BGP in a central box.” The value is the ability to combine network-wide state with the perspective of the router that actually needs the decision. That requires accurate topology visibility. If the controller is stale, the centralised decision process can be consistently wrong even though BGP sessions and basic reachability remain healthy.

ENGINEERING ORIENTATION · 03 / INVESTIGATE

A route looks valid—but the chosen exit is wrong

Explore the evidence an engineer should check before changing routing policy.

Check session health firstAn established BGP session confirms that the peer relationship is up; it does not prove the selected exit is optimal for this router.
Section 3 · Investigate the incident

Why Did One Region Get the Wrong Exit?

A BGP session can be established, a destination prefix can be present, and traffic can still take an unexpected path. When a Routing Control Platform uses topology information to calculate route decisions, an engineer must investigate both the routing inputs and the state from which those decisions were derived.

The critical skill is separating a symptom from its cause. An unexpected exit does not automatically mean that BGP is broken, that the destination is unreachable, or that LOCAL_PREF needs to be changed. The route may have been calculated using an outdated view of the internal network.

Incident hypothesis

One region prefers Exit B even though Exit A is closer according to the current IGP topology. BGP remains established and the destination is available. The controller's topology snapshot is several minutes old. Treat stale state as a hypothesis to verify—not as a conclusion to assume.

3.1 Start with the observed symptom

Imagine a network with two external exits advertising the same destination. The West region currently forwards through Exit B. An engineer's current IGP measurements show a lower cost to Exit A, but the controller still reports Exit B as the selected path.

OBSERVED
Unexpected exit selection

Traffic from West uses Exit B even though current topology measurements suggest Exit A is closer. Confirm the actual forwarding path instead of relying only on a dashboard summary.

KNOWN EVIDENCE
BGP is still established

The relevant BGP neighbour is up and the destination prefix is available. This makes a completely failed BGP session less likely, but does not rule out route-policy or path-distribution problems.

TOPOLOGY
Current costs differ

West's measured IGP costs are 18 to Exit A and 61 to Exit B. These figures are evidence about internal reachability—not proof that BGP must choose Exit A.

SUSPICIOUS STATE
Controller data is stale

The controller's topology snapshot is reported to be nine minutes old. Verify that timestamp and compare it with the actual IGP state and recent topology changes.

3.2 Follow the evidence in a deliberate order

Use a consistent investigation sequence to avoid changing policy before the cause is known. Each check should either support or eliminate a plausible failure mode.

01 · Confirm Reproduce the wrong exit and identify the affected router, prefix and traffic direction.
02 · Inspect Check BGP neighbour state, candidate paths, attributes and route availability.
03 · Compare Compare current IGP costs and next-hop reachability with the controller's recorded view.
04 · Validate Reconcile the topology, recompute the decision and verify the router's installed route and traffic.

3.3 Distinguish the possible causes

Possible cause Evidence to collect What it tells you
BGP session failure Neighbour state, logs, route withdrawal and session history. A down session can affect route availability, but an established session does not prove that the selected path is correct.
Missing route or next hop Received prefix, next-hop resolution, RIB and FIB entries. The route may be unavailable or unusable even when other BGP sessions remain up.
Policy or BGP attributes LOCAL_PREF, AS_PATH, MED, import/export policy and candidate-path visibility. A policy decision may legitimately override an engineer's expectation based on IGP distance alone.
Stale controller topology Snapshot timestamp, topology events, current IGP database and controller refresh status. The controller may be calculating from an older network state than the routers are using.
Distribution or installation problem Computed decision, advertised route, received route and local FIB. The intended decision may not have reached the router or may not have been installed as expected.

3.4 Why stale topology can produce a wrong decision

A topology-aware controller can only calculate from the information it has. If a link changes, a metric is updated or a failure occurs, the controller needs to learn the change and update its model. Until that happens, the controller's estimate of the path to an exit may no longer match the actual network.

This is a consistency problem between the observed network and the model used for route computation. The controller might produce a decision that was reasonable under the old topology but is no longer appropriate under the current one.

The effect depends on the implementation and the failure. A stale snapshot does not always cause an incorrect exit, a forwarding loop or a blackhole. It becomes operationally significant when the outdated information changes the computed decision or leaves a next hop unreachable.

3.5 Remediate the cause, not just the symptom

If evidence confirms that the topology view is stale, the corrective action is to restore accurate information and recalculate the route decision using the platform's supported recovery procedure. Avoid applying a blanket LOCAL_PREF change simply to force the desired exit: that may mask the issue and create unintended results for other regions or destinations.

  • Refresh: Restore topology collection or synchronisation and confirm that the controller has received the current state.
  • Recompute: Recalculate the route decision with current topology, BGP routes and policy.
  • Redistribute: Confirm that the intended routing update reaches the correct router or client.
  • Verify installation: Inspect the selected route, next hop and local FIB.
  • Verify service: Test the actual traffic path and confirm reachability and performance meet expectations.

3.6 What counts as proof that the incident is resolved?

A green controller status or a successful recalculation is not sufficient on its own. The investigation is complete when the inputs are current, the selected route matches the intended policy and topology, the router installs the expected forwarding entry, and traffic follows the intended path.

Section 3 engineering takeaway

Investigate unexpected routing by correlating BGP state, route attributes, current IGP costs, controller topology freshness, route distribution and FIB state. When the controller's view is stale, repair and verify the data pipeline before changing routing policy. The incident lab that follows lets you test this reasoning against the West-region scenario.

ENGINEERING ORIENTATION · 04 / CORRELATE & OPERATE

Recovery is not proven by a single green status

Select the evidence you have verified. A routing change is complete only when the control-plane decision and forwarding outcome agree.

Evidence verified: 0 of 4
Engineering takeaway: correlate topology, route selection, forwarding state and observed traffic. A correct BGP decision alone does not guarantee end-to-end service recovery.

Correlate and Operate: Validate the Routing Decision

A routing decision is not proven correct simply because a controller calculated a path or a router received an update. Engineers need to connect the network's topology, BGP information, policy decision, installed forwarding state and observed traffic into one evidence chain.

That is the operational challenge behind a Routing Control Platform (RCP). Its value depends not only on the decision logic, but also on the freshness and consistency of the information used to make decisions—and on whether the resulting routes are distributed and installed as intended.

Follow the evidence across the control and data planes

When a region takes an unexpected exit, investigate the complete sequence. A controller may have selected a valid route using an outdated topology snapshot; a router may have received the intended route but rejected it through policy; or the routing table may be correct while forwarding still fails because of a next-hop or downstream connectivity problem.

01
Topology and reachability Check IGP state, link costs, BGP sessions and reachable prefixes.
02
Decision and policy Confirm route attributes, policy inputs, router perspective and calculation time.
03
Distribution and installation Check the advertised route, receiving router's RIB and installed FIB entry.
04
Traffic and stability Test the real path, service outcome, convergence and post-change behaviour.

Correlate symptoms with the right evidence

Observed symptom Evidence to correlate Engineering interpretation
Unexpected exit selected IGP costs, BGP attributes, policy, topology timestamp and router perspective The choice may reflect policy or a different network view—not necessarily a failed session.
Route missing from a router Received updates, import policy, next-hop reachability and routing table The route may be unavailable, filtered, ineligible or not installed.
Route present, traffic fails FIB entry, resolved next hop, interface state, ACLs and end-to-end probes Control-plane reachability does not by itself prove successful packet delivery.
Repeated path changes Update history, attribute changes, topology events, timers and policy interactions Look for interacting decisions or unstable inputs before changing attributes.

Understand the mechanisms around route selection

Hot-potato routing

Hot-potato routing generally favours the closest eligible exit according to the router's IGP view after higher-priority BGP decision criteria have been considered. Two routers can legitimately choose different exits because their IGP costs differ. A controller's view must therefore match the intended decision perspective; a low IGP cost alone does not override every BGP attribute or policy.

Multipath and BGP Add-Path

BGP multipath can allow multiple eligible paths to be installed or used, subject to implementation and configuration. BGP Add-Path allows multiple paths for a prefix to be advertised to a peer, helping reduce some path-visibility limitations. Neither feature automatically guarantees a better path or removes the need to validate policy, next-hop resolution and forwarding behaviour.

Next-hop tracking and MED

Next-hop tracking helps a router reassess route eligibility when next-hop reachability changes. MED can influence selection between routes from the same neighbouring AS, depending on the comparison rules and policy. In complex designs, interacting MED values, route reflection and changing topology can contribute to repeated path changes—but oscillation is not inevitable. Examine the actual attributes, decision order and update sequence before attributing a problem to MED.

Design boundary: RCP is a historically important control-plane architecture, not a universal modern product or a guaranteed replacement for Route Reflectors. Any centralised or controller-assisted design must account for failure modes, topology freshness, policy consistency, route distribution, controller redundancy and safe fallback behaviour.

A safe recovery workflow

Once the evidence points to stale or inconsistent topology, avoid making unrelated policy changes just to force a preferred exit. Correct the underlying input first, then confirm each downstream stage.

DetectReconcileRecomputeRedistributeVerify

  1. Capture the baseline. Record the affected prefix, selected exit, IGP costs, BGP attributes, relevant updates and current RIB/FIB state.
  2. Reconcile the network view. Confirm that topology data and reachability reflect current link and routing state. Check timestamps and the source of the data.
  3. Recompute and distribute. Re-evaluate the route using the intended policy and perspective, then verify that the expected decision reaches the affected routers.
  4. Verify installation. Confirm the route in the router's RIB, resolve the next hop and check the corresponding FIB entry. A received update is not the same as an installed route.
  5. Test the service and monitor. Use appropriate path probes or traffic tests, watch for unexpected updates or route churn, and retain a rollback path if the change causes regressions.

How do you know the incident is resolved?

Recovery is complete only when the intended route decision is visible, the forwarding entry is installed, the next hop is reachable, traffic behaves as expected and the system remains stable after convergence. Record the before-and-after evidence so that the fix can be explained and repeated—not merely assumed from a green status indicator.

Engineering takeaway Treat routing as an evidence chain: topology and reachability → BGP attributes and policy → route decision → route distribution → RIB/FIB installation → observed traffic. Verify every stage before declaring recovery.

Engineering conclusion · Routing control

From Centralised Routing Intelligence to Verified Forwarding

The Routing Control Platform concept highlights a fundamental challenge in BGP engineering: a route-selection decision is only as reliable as the network information, policy and router perspective behind it.

By combining BGP reachability with IGP topology information, an RCP can calculate routing decisions with a broader view of the network. The operational challenge is ensuring that this view remains accurate, the intended decisions reach the relevant routers, and the resulting forwarding state delivers the expected outcome.

Four engineering principles to take away

01 · Understand the architecture Separate topology discovery, BGP route information, route computation and packet forwarding.
02 · Compare perspectives Account for route visibility, IGP costs, BGP attributes and policy before explaining an exit choice.
03 · Investigate with evidence Distinguish session failure, missing reachability, policy effects, stale topology and installation problems.
04 · Verify the outcome Confirm the selected route, installed forwarding entry, next-hop reachability, traffic behaviour and stability.

The broader lesson for software-defined networking

RCP is a useful architectural case study in separating routing intelligence from packet forwarding. Its ideas remain relevant when evaluating controller-assisted networking, centralised policy, topology-aware path computation and modern network automation. However, an architecture should be judged against its actual implementation, failure handling, scalability and operational requirements—not treated as a universal replacement for established BGP designs.

The engineer's final test is not whether the controller calculated a route. It is whether the intended route was installed, traffic followed the expected path, and the network remained stable.

RCP vs Route Reflector: Which Exit Does Each Router Choose?

Experiment with the network perspective, IGP costs and routing policy. Then run the decision engine to see how a traditional Route Reflector and an RCP can produce different exit choices.

Interactive Lab
Ready: Change the conditions and run the decision engine.
Route Reflector Shared decision
Selected Exit Exit A
The RR makes the best-path decision from its own reference point and reflects that selected route to clients.
RR → Exit A 20
RR → Exit B 40
Result delivered to client Exit A
Routing Control Platform Per-router decision
Selected Exit Exit A
The RCP evaluates the available exits using the selected router's network perspective.
Router → Exit A 20
Router → Exit B 65
Decision comparison Same as RR
Routing Perspective Awaiting calculation
SOURCE WEST
RR1 RR view
ROUTER WEST VIEW
EXIT A EGRESS
EXIT B EGRESS
A: 20
B: 40
RR-selected path
Exit A
Exit B
Engineering Insight

The important difference is network perspective. A Route Reflector provides scalability by reflecting selected routes, while an RCP can calculate the appropriate route for the individual router using broader topology visibility.

☕
Support Network Insight
Help support the development of interactive networking labs, BGP simulations, and educational content.
Support the Project