MPLS

Segment Routing – Introduction

by Blog

Segment Routing

In today's interconnected world, where data traffic is growing exponentially, network operators face numerous challenges regarding scalability, flexibility, and efficiency. To address these concerns, segment routing has emerged as a powerful networking paradigm that offers a simplified and programmable approach to traffic engineering. In this blog post, we will explore the concept of segment routing, its benefits, and its applications in modern networks.

Segment routing is a forwarding paradigm that leverages source routing principles to steer packets along a predetermined path through a network. Instead of relying on complex routing protocols and their associated overhead, segment routing enables the network to be programmed with predetermined instructions, known as segments, to define the path packets should traverse. These segments can represent various network resources, such as links, nodes, or services, and are encoded in the packet's header.

Enhanced Network Scalability: Segment routing enables network operators to scale their networks effortlessly. By leveraging existing routing mechanisms and avoiding the need for extensive protocol exchanges, segment routing simplifies network operations, reduces overhead, and enhances scalability.

Traffic Engineering and Optimization: With segment routing, network operators gain unparalleled control over traffic engineering. By specifying explicit paths for packets, they can optimize network utilization, avoid congestion, and prioritize critical applications, ensuring a seamless user experience.

Fast and Efficient Network Restoration: Segment routing's inherent flexibility allows for rapid network restoration in the event of failures. By dynamically rerouting traffic along precomputed alternate paths, segment routing minimizes downtime and enhances network resilience.

Matt Conran

Highlights: Segment Routing

What is Segment Routing?

Segment Routing is a forwarding paradigm that leverages the concept of source routing. It allows network operators to define a path for network traffic by encoding instructions into the packet header itself. This eliminates the need for complex routing protocols, simplifying network operations and enhancing scalability.

Traditional routing protocols often struggle with complexity and scalability, leading to increased operational costs and reduced network performance. Segment Routing addresses these challenges by simplifying the data path and enhancing traffic engineering capabilities, making it an attractive option for Internet Service Providers (ISPs) and enterprises alike.

**Source-based routing technique**

Note: Source-Based Routing

Segment routing is a source-based routing technique that enables a source node to define the path that a packet will take through the network. This is achieved by encoding a list of instructions, or “segments,” into the packet header.

Each segment represents a specific instruction, such as forwarding the packet to a particular node or applying a specific service. By doing so, segment routing eliminates the need for complex protocols like MPLS and reduces the reliance on network-wide signaling, leading to a more efficient and manageable network.

1- ) At its core, Segment Routing is a source-based routing technique that enables packets to follow a specified path through the network. Unlike conventional routing methods that rely on complex signaling protocols and state information, SR uses a list of segments embedded within packets to dictate the desired path.

2- ) These segments represent various waypoints or instructions that guide packets from source to destination, allowing for more efficient and predictable routing.

3- ) Segment Routing is inherently compatible with both MPLS (Multiprotocol Label Switching) and IPv6 networks, making it a versatile choice for diverse network environments. By leveraging existing infrastructure, SR minimizes the need for significant hardware upgrades, facilitating a smoother transition for network operators.

Key Points:

Scalability and Flexibility: Segment Routing provides enhanced scalability by allowing network operators to define paths dynamically based on network conditions. It enables traffic engineering, load balancing, and fast rerouting, ensuring efficient resource utilization and optimal network performance.

Simplified Operations: By leveraging source routing, Segment Routing simplifies network operations. It eliminates the need for maintaining and configuring multiple protocols, reducing complexity and operational overhead. This results in improved network reliability and faster deployment of new services.

**Implementing Segment Routing in Your Network**

Transitioning to Segment Routing involves several key steps. First, network operators need to assess their existing infrastructure to determine compatibility with SR. This may involve updating software, configuring routers, and integrating SR capabilities into the network.

Once the groundwork is laid, operators can begin defining segments and policies to guide packet flows. This process involves careful planning to ensure optimal performance and alignment with business objectives. By leveraging automation tools and intelligent analytics, operators can further streamline the implementation process and continuously monitor network performance.

Applications of Segment Routing

Traffic Engineering: Segment Routing enables intelligent traffic engineering by allowing operators to specify explicit paths for traffic flows. This empowers network operators to optimize network utilization, minimize congestion, and provide quality-of-service guarantees.

Network Slicing: Segment Routing facilitates network slicing, a technique that enables the creation of multiple virtual networks on a shared physical infrastructure. By assigning unique segments to each slice, Segment Routing ensures isolation, scalability, and efficient resource allocation.

5G Networks: Segment Routing plays a crucial role in the evolution of 5G networks. It enables network slicing, network function virtualization, and efficient traffic engineering, providing the necessary foundation for the deployment of advanced 5G services and applications.

Complexity and Scale of MPLS Networks

– The complexity and scale of MPLS networks have grown over the past several years. Segment routing simplifies the propagation of tags through an extensive network by reducing overhead communication or control-plane traffic.

– By ensuring accurate and timely control-plane communication, traffic can reach its destination properly and efficiently. As a second-order effect, this reduces the likelihood of user error, order of operation errors, and control-plane sync issues in the network.

– Segment Routing’s architecture allows the software to define traffic flow proactive rather than reactively responding to network issues. With these optimizations, Segment Routing has become a hot topic for service providers and enterprises alike, and many are migrating from MPLS to Segment Routing.

**The Architecture of MPLS**

At its core, MPLS operates by assigning labels to data packets, allowing routers to make forwarding decisions based on these labels rather than the traditional, more cumbersome IP addresses. This label-based system not only streamlines the routing process but also significantly enhances the speed and efficiency of data transmission.

The architecture of MPLS involves several key components, including Label Edge Routers (LERs) and Label Switch Routers (LSRs), which work together to ensure seamless data flow across the network. Understanding the roles of these components is crucial for grasping how MPLS networks maintain their robustness and scalability.

**Challenges in Implementing MPLS**

Despite its numerous advantages, deploying MPLS networks comes with its own set of challenges. The initial setup and configuration can be complex, requiring specialized knowledge and skills. Furthermore, maintaining and troubleshooting MPLS networks demand continuous monitoring and management to ensure optimal performance.

Security is another concern, as the complexity of MPLS can sometimes obscure potential vulnerabilities. Organizations must implement robust security measures to protect their MPLS networks from threats and ensure the integrity of their data.

Example Technology: MPLS Forwarding & LDP

### What is MPLS Forwarding?

MPLS forwarding is a method of data transport that simplifies and accelerates the routing process. Unlike traditional IP routing, which uses destination IP addresses to forward packets, MPLS uses labels. Each packet gets a label, which is a short identifier that routers use to determine the packet’s next hop. This approach reduces the complexity of routing tables and increases the speed at which data is processed. MPLS is particularly beneficial for building scalable, high-performance networks, making it a preferred choice for service providers and large enterprises.

### The Role of LDP in MPLS Networks

The Label Distribution Protocol (LDP) is integral to the operation of MPLS networks, as it manages the distribution of labels between routers. LDP establishes label-switched paths (LSPs), which are the routes that data packets follow through an MPLS network. By automating the process of label distribution, LDP ensures efficient and consistent communication between routers, thus enhancing network reliability and performance. Understanding LDP is crucial for network engineers looking to implement or manage MPLS systems.

### Advantages of Using MPLS and LDP

The combination of MPLS and LDP offers numerous advantages. Firstly, MPLS can handle multiple types of traffic, such as IP packets, ATM, and Ethernet frames, making it versatile and adaptable. The use of labels instead of IP addresses results in faster data forwarding, reducing latency and improving network speed. Additionally, MPLS supports Quality of Service (QoS), allowing for prioritization of critical data, which is essential for applications like VoIP and video conferencing. LDP enhances these benefits by providing a robust framework for label management, ensuring network stability and efficiency.

Segment Routing –  A Forwarding Paradigm

Segment routing is a forwarding paradigm that allows network operators to define packet paths by specifying a series of segments. These segments represent instructions that guide the packet’s journey through the network. Network engineers can optimize traffic flow, improve network scalability, and provide advanced services by leveraging segment routing.

**The Mechanics of Segment Routing**

At its core, Segment Routing leverages the source-routing principle. Instead of relying on each router to make forwarding decisions, SR allows the source node to encode the path a packet should take through the network. This path is defined by an ordered list of segments, which can be topological or service-based. By eliminating the need for complex signaling protocols, SR reduces overhead and simplifies the network infrastructure.

Recap: Segment routing leverages the concept of source routing, where the sender of a packet determines the complete path it will take through the network. Routers can effortlessly steer packets along a predefined path by assigning a unique identifier called a segment to each network hop, avoiding complex routing protocols and reducing network overhead.

Key Concepts – Components

To fully grasp segment routing, it’s essential to familiarize yourself with its key concepts and components. One fundamental element is the Segment Identifier (SID), which represents a specific network node or function. SIDs are used to construct explicit paths and enable traffic engineering. Another essential concept is the label stack, which allows routers to stack multiple SIDs together to form a forwarding path. Understanding these concepts is crucial for effectively implementing segment routing in network architectures.

**The Building Blocks**

  • Segment IDs: Segment IDs are fundamental elements in segment routing. They uniquely identify a specific segment within the network. Depending on the network infrastructure, these IDs can be represented by various formats, such as IPv6 addresses or MPLS labels.
  • Segment Routing Headers: Segment routing headers contain the segment instructions for packets. They are added to the packet’s header, indicating the sequence of segments to traverse. These headers provide the necessary information for routers to make forwarding decisions based on the defined segments.

**Traffic Engineering with Segment Routing**

  • Traffic Steering: Segment routing enables precise traffic steering capabilities, allowing network operators to direct packets along specific paths based on their requirements. This fine-grained control enhances network efficiency and enables better utilization of network resources.
  • Fast Reroute: Fast Reroute (FRR) is a crucial feature of segment routing that enhances network resiliency. By leveraging backup paths and pre-calculated segments, segment routing enables rapid traffic rerouting in case of link failures or congestion. This ensures minimal disruption and improved quality of service for critical applications.

**Integration with Network Services**

  • Service Chaining: Segment routing seamlessly integrates with network services, enabling efficient service chaining. Service chaining directs traffic through a series of service functions, such as firewalls or load balancers, in a predefined order. With segment routing, this process becomes more streamlined and flexible.
  • Network slicing: Network slicing leverages the capabilities of segment routing to create virtualized networks within a shared physical infrastructure. It enables the provisioning of isolated network slices tailored to specific requirements, guaranteeing performance, security, and resource isolation for different applications or tenants.

Understanding MPLS

MPLS, a versatile protocol, has been a stalwart in the networking industry for decades. It enables efficient packet forwarding by leveraging labels attached to packets for routing decisions. MPLS provides benefits such as traffic engineering, Quality of Service (QoS) control, and Virtual Private Network (VPN) support. Understanding the fundamental concepts of label switching and label distribution protocols is critical to grasping MPLS.

The Rise of Segment Routing

Segment Routing, on the other hand, is a relatively newer paradigm that simplifies network architectures and enhances flexibility. It leverages the concept of source routing, where the source node explicitly defines the path that packets should traverse through the network. By incorporating this approach, segment routing eliminates the need to maintain per-flow state information in network nodes, leading to scalability improvements and more accessible network management.

Key Differences and Synergies

While MPLS and Segment Routing have unique characteristics, they can also complement each other in various scenarios. Understanding the differences and synergies between these technologies is crucial for network architects and operators. MPLS offers a wide range of capabilities, including Traffic Engineering (MPLS-TE) and VPN services, while Segment Routing simplifies network operations and offers inherent traffic engineering capabilities.

MPLS and BGP-free Core

So, what is segment routing? Before discussing a segment routing solution and the details of segment routing vs. MPLS, let us recap how MPLS works and the protocols used. MPLS environments have both control and data plane elements.

A BGP-free core operates at network edges, participating in full mesh or route reflection design. BGP is used to pass customer routes, Interior Gateway Protocol (IGP) to pass loopbacks, and Label Distribution Protocol (LDP) to label the loopback.

Example BGP Technology: BGP Route Reflection

**The Challenge of Scalability in BGP**

In large networks, maintaining a full mesh of BGP peering relationships becomes impractical. As the number of routers increases, the number of required connections grows exponentially. This complexity can lead to increased configuration errors, higher resource consumption, and slower convergence times, making network management a daunting task.

**Enter BGP Route Reflection**

BGP Route Reflection is a clever solution to the scalability challenges of traditional BGP. By designating certain routers as route reflectors, network administrators can significantly reduce the number of BGP sessions required. These reflectors act as intermediaries, receiving routes from clients and redistributing them to other clients without the need for a full mesh.

**Benefits of BGP Route Reflection**

Implementing BGP Route Reflection offers numerous advantages. It simplifies network topology, reduces configuration complexity, and lowers the resource demands on routers. Additionally, it enhances network stability by minimizing the risk of configuration errors and improving convergence times during network changes or failures.

Labels and BGP next hops

LDP or RSVP establishes MPLS label-switched paths ( LSPs ) throughout the network domain. Labels are assigned to the BGP next hops on every router where the IGP in the core provides reachability for remote PE BGP next hops.

As you can see, several control plane elements interact to provide complete end-to-end reachability. Unfortunately, the control plane is performed hop-by-hop, creating a network state and the potential for synchronization problems between LDP and IGP.

Related: Before you proceed, you may find the following post helpful:

  1. Observability vs Monitoring
  2. Network Traffic Engineering
  3. What Is VXLAN
  4. Technology Insight for Microsegmentation
  5. WAN SDN

Segment Routing

Keep Complexity to Edges

In 2002, the IETF published RFC 3439, an Internet Architectural Guideline and Philosophy. It states, “In short, the complexity of the Internet belongs at the edges, and the IP layer of the Internet should remain as simple as possible.” When applying this concept to traditional MPLS-based networks, we must bring additional network intelligence and enhanced decision-making to network edges. Segment Routing is a way to get intelligence to the edge and Software-Defined Networking (SDN) concepts to MPLS-based architectures.

MPLS-based architectures:

MPLS, or Multiprotocol Label Switching, is a versatile networking technology that enables the efficient forwarding of data packets. Unlike traditional IP routing, MPLS utilizes labels to direct traffic along predetermined paths, known as Label Switched Paths (LSPs). This label-based approach offers enhanced speed, flexibility, and traffic engineering capabilities, making it a popular choice for modern network infrastructures.

Components of MPLS-Based Architectures

It is crucial to understand the workings of MPLS-based architectures’ key components. These include:

1. Label Edge Routers (LERs): LERs assign labels to incoming packets and forward them into the MPLS network.

2. Label Switch Routers (LSRs): LSRs form the core of the MPLS network, efficiently switching labeled packets along the predetermined LSPs.

3. Label Distribution Protocol (LDP): LDP facilitates the exchange of label information between routers, ensuring the proper establishment of LSPs.

Guide on a BGP-free core.

Here, we have a typically pre-MPLS setup. The main point is that the P node is only running OSPF. It does not know the CE routers or any other BGP routes. Then, BGP runs across a GRE tunnel to the CE nodes. The GRE tunnel we are running is point-to-point.

When we run a traceroute from CE1 to CE2, the packets traverse the GRE tunnel, and no P node interfaces are in the trace. The main goal here is to free up resources in the core, which is the starting point of MPLS networking. In the lab guide below, we will upgrade this to MPLS.

overlay networking

Source Packet Routing

Segment routing is a development of the Source Packet Routing in the Network (SPRING) working group of the IETF. The fundamental idea is the same as Service Function Chaining (SFC). Still, rather than assuming the processes along the path will manage the service chain, Segment Routing considers the routing control plane to handle the flow path through a network.

Segment routing (SR) is a source-based routing technique that streamlines traffic engineering across network domains. It removes network state information from transit routers and nodes and puts the path state information into packet headers at an ingress node.

segment routing solution
Diagram: Issues with MPLS and the need for a segment routing solution.

MPLS Traffic Engineering

MPLS TE is an extension of MPLS, a protocol for efficiently routing data packets across networks. It provides a mechanism for network operators to control and manipulate traffic flow, allowing them to allocate network resources effectively. MPLS TE utilizes traffic engineering to optimize network paths and allocate bandwidth based on specific requirements.

It allows network operators to set up explicit paths for traffic, ensuring that critical applications receive the necessary resources and are not affected by congestion or network failures. MPLS TE achieves this by establishing Label Switched Paths (LSPs) that bypass potential bottlenecks and follow pre-determined routes, resulting in a more efficient and predictable network.

Guide on MPLS TE

In this lab, we will examine MPLS TE with ISIS configuration. Our MPLS core network comprises PE1, P1, P2, P3, and PE2 routers. The CE1 and CE2 routers use regular IP routing. All routers are configured to use IS-IS L2. 

There are four main items we have to configure:

  • Enable MPLS TE support:
    • Globally
    • Interfaces
  • Configure IS-IS to support MPLS TE.
  • Configure RSVP.
  • Configure a tunnel interface.
MPLS TE
Diagram: MPLS TE

**Synchronization Problems**

Packet loss can occur in two scenarios when the actions of IGP and LDP are not synchronized. Firstly, when an IGP adjacency is established, the router begins to forward packets using the new adjacency before the actual LDP exchange occurs between peers on that link.

Secondly, when an LDP session terminates, the router forwards traffic using the existing LDP peer link. This issue is resolved by implementing network kludges and turning on auto-synchronization between IGP and LDP. Additional configurations are needed to get these two control planes operating safely.

Solution – Segment Routing

Segment Routing is a new architecture built with SDN in mind. Separating data from the control plane is all about network simplification. SDN is a great concept; we must integrate it into today’s networks. The SDN concept of simplification is a driver for introducing Segment Routing.

Segment routing vs MPLS

Segment routing utilizes the basics of MPLS but with fewer protocols, less protocol interaction, and less state. It is also applied to MPLS architecture with no change to the forwarding plane. Existing devices switching based on labels may only need a software upgrade. The virtual overlay network concept is based on source routing. The source chooses the path you take through the network. It steers a packet through an ordered list of instructions called segments.

Like MPLS, Segment Routing is based on label switching without LDP or RSVP. Labels are called segments, and we still have push, swap, and pop actions. You do not keep the state in the middle of the network, as the state is in the packet instead. In the packet header, you put a list of segments. A segment is an instruction – if you want to go to C, use A-B-C.

  • With Segment Routing, the Per-flow state is only maintained at the ingress node to the domain.

It is all about getting a flow concept, mapping it to a segment, and putting that segment on a true path. It keeps the properties of resilience ( fast reroute) but simplifies the approach with fewer protocols. As a result, it provides enhanced packet forwarding behavior while minimizing the need to maintain the network state.

Guide on MPLS forwarding.

The previous lab guide can easily be upgraded to MPLS. We removed the GRE tunnel and the iBGP neighbors. MPLS is enabled with the mpls ip command on all interfaces on the P node and the PE node interfaces facing the P node. Now, we have MPLS forwarding based on labels while maintaining a BGP-free core. Notice how the two CEs can ping each other, and there is no route for 5.5.5.5 in the P node.

MPLS forwarding
Diagram: MPLS forwarding

Two types of initial segments are defined

Node and Adjacency

1- Nodel label: Nodel label is globally unique to each node. For example, a node labeled “Dest” has label 65 assigned to it, so any ingress network traffic with label 65 goes straight to Dest. By default, it will take the best path.

2- Then we have the Adjacency label: a locally significant label that takes packets to an adjacent path. It forces packets through a specific link and offers more specific path forwarding than a nodel label.

segment routing vs mpls
Diagram: Segment routing vs MPLS and the use of labels.

Segment routing: A new business model

Segment Routing addresses current issues and brings a new business model. It aims to address the pain points of existing MPLS/IP networks in terms of simplicity, scale, and ease of operation. Preparing the network with an SDN approach allows application integration directly on top of it.

Segment Routing allows you to take certain traffic types and make a routing decision based on that traffic class. It permits you to bring traffic that you think is important, such as Video or Voice, to go a different way than best efforts traffic.

Traffic paths can be programmed end-to-end for a specific class of customer. It moves away from the best-path model by looking at the network and deciding on the source. It is very similar to MPLS, but you use the labels differently.

SDN controller & network intelligence

Controller-based networks sit perfectly with this technology. It’s a very centralized and controller application methodology. The SDN controller gathers network telemetry information, decides based on a predefined policy, and pushes information to nodes to implement data path forwarding. Network intelligence such as link utilization, path response time, packet drops, latency, and jitter are extracted from the network and analyzed by the controller.

The intelligence now sits at the edges. The packet takes a path based on the network telemetry information extracted by the controller. The result is that the ingress node can push a label stack to the destination to take a specific path.

Your chosen path at the network’s edge is based on telemetry information.

Recap: Applications of Segment Routing:

1. Traffic Engineering and Load Balancing: Segment routing enables network operators to dynamically steer traffic along specific paths to optimize network resource utilization. This capability is handy in scenarios where certain links or nodes experience congestion, enabling network operators to balance the load and efficiently utilize available resources.

2. Service Chaining: Segment routing allows for the seamless insertion of network services, such as firewalls, load balancers, or WAN optimization appliances, into the packet’s path. By specifying the desired service segments, network operators can ensure traffic flows through the necessary services while maintaining optimal performance and security.

3. Network Slicing: With the advent of 5G and the proliferation of the Internet of Things (IoT) devices, segment routing can enable efficient network slicing. Network slicing allows for virtualized networks, each tailored to the specific requirements of different applications or user groups. Segment routing provides the flexibility to define and manage these virtualized networks, ensuring efficient resource allocation and isolation.

Segment Routing: Closing Points

Segment routing offers a promising solution to the challenges faced by modern network operators. Segment routing enables efficient and optimized utilization of network resources by providing simplified network operations, enhanced scalability, and traffic engineering flexibility.

With its applications ranging from traffic engineering to service chaining and network slicing, segment routing is poised to play a crucial role in the evolution of modern networks. As the demand for more flexible and efficient networks grows, segment routing emerges as a powerful tool for network operators to meet these demands and deliver a seamless and reliable user experience.

Summary: Segment Routing

Segment Routing, also known as SR, is a cutting-edge technology that has revolutionized network routing in recent years. This innovative approach offers numerous benefits, including enhanced scalability, simplified network management, and efficient traffic engineering. This blog post delved into Segment Routing and explored its key features and advantages.

Understanding Segment Routing

Segment Routing is a flexible and scalable routing paradigm that leverages source routing techniques. It allows network operators to define a predetermined packet path by encoding it in the packet header. This eliminates the need for complex routing protocols and enables simplified network operations.

Key Features of Segment Routing

Traffic Engineering:

Segment Routing provides granular control over traffic paths, allowing network operators to steer traffic along specific paths based on various parameters. This enables efficient utilization of network resources and optimized traffic flows.

Fast Rerouting:

One notable advantage of Segment Routing is its ability to quickly reroute traffic in case of link or node failures. With the predefined paths encoded in the packet headers, the network can dynamically reroute traffic without relying on time-consuming protocol convergence.

Network Scalability:

Segment Routing offers excellent scalability by leveraging a hierarchical addressing structure. It allows network operators to segment the network into smaller domains, simplifying management and reducing the overhead associated with traditional routing protocols.

Use Cases and Benefits

Service Provider Networks:

Segment Routing is particularly beneficial for service provider networks. It enables efficient traffic engineering, seamless service provisioning, and simplified network operations, leading to improved quality of service and reduced operational costs.

Data Center Networks:

In data center environments, Segment Routing offers enhanced flexibility and scalability. It enables optimal traffic steering, efficient workload balancing, and simplified network automation, making it an ideal choice for modern data centers.

Conclusion:

In conclusion, Segment Routing is a powerful and flexible technology that brings numerous benefits to modern networks. Its ability to provide granular control over traffic paths, fast rerouting, and network scalability makes it an attractive choice for network operators. As Segment Routing continues to evolve and gain wider adoption, we can expect to see even more innovative use cases and benefits in the future.

ip routing

Advances of IP routing and Cloud

by Blog

 

ip routing

 

With the introduction and hype around Software Defined Networking ( SDN ) and Cloud Computing, one would assume that there has been little or no work with the advances in IP routing. You could say that the cloud has clouded the mind of the market. Regardless of the hype around this transformation, routing is still very much alive and makes up a vital part of the main internet statistics you can read. Packets still need to get to their destinations.

 

Advanced in IP Routing

The Internet Engineering Task Force (IETF) develops and promotes voluntary internet standards, particularly those that comprise the Internet Protocol Suite (TCP/IP). The IETF shapes what comes next, and this is where all the routing takes place. It focuses on anything between the physical layer and the application layer. It doesn’t focus on the application itself, but on the technologies used to transport it, for example, HTTP.

In the IETF, no one is in charge, anyone can contribute, and everyone can benefit. As you can see from the chart below, that routing ( RTG ) has over 250 active drafts and is the most popular working group within the IETF.

 

 

IP routinng
Diagram: IETF Work Distribution.

 

The routing area is responsible for ensuring the continuous operation of the Internet routing system by maintaining the scalability and stability characteristics of the existing routing protocols and developing new protocols, extensions, and bug fixes promptly

The following table illustrates the subgroups of the RTG working group:

Bidirectional Forwarding Detection (BFD) Open Shortest Path First IGP (OSPF)
Forwarding and Control Element Separation (forces) Routing Over Low power and Lossy networks (roll)
Interface to the Routing System (i2rs) Routing Area Working Group (RTGW)
Inter-Domain Routing (IDR) Secure Inter-Domain Routing (SCIDR)
IS-IS for IP Internets (isis) Source Packet Routing in Networking (spring)
Keying and Authentication for Routing Protocols (Karp)
Mobile Ad-hoc Networks (manet)

The chart below displays the number of drafts per subgroup of the routing area. There has been a big increase in the subgroup “roll,” which is second to BGP. “Roll” relates to “Routing Over Low power and Lossy networks” and is driven by the Internet of Everything and Machine-to-Machine communication.

 

IP ROUTING
Diagram: RTG Ongoing Work.

 

OSPF Enhancements

OSPF is a link-state protocol that uses a common database to determine the shortest path to any destination.

Two main areas of interest in the Open Shortest Path First IGP (OSPF) subgroups are OSPFv3 LSA Extendibility and Remote Loop-Free Alternatives ( LFAs ). One benefit IS-IS has over OSPF is its ability to easily introduce new features with the inclusion of Type Length Values ( TLVs ) and sub-TLVs. The IETF draft-IETF-OSPF-ospfv3-lsa-extend extends the LSA format by allowing the optional inclusion of TLVs, making OSPF more flexible and extensible. For example, OSPFv3 uses a new TLV to support intra-area Traffic Engineering ( TE ), while OSPFv2 uses an opaque LSA.

 

TLV for OSPFv3
Diagram: TLV for OSPFv3.

 

Another shortcoming of OSPF is that it does not install a backup route in the routing table by default. Having a pre-installed backup up path greatly improves convergence time. With pre-calculated backup routes already installed in the routing table, the router process does not need to go through the convergence process’s LSA propagation and SPF calculation steps.

 

Loop-free alternatives (LFA)

Loop-Free Alternatives ( LFA ), known as Feasible Successors in EIGRP, are local router decisions to pre-install a backup path.
In the diagram below:

-Router A has a primary ( A-C) and secondary ( A-B-C) path to 10.1.1.0/24
-Link State allows Router A to know the entire topology
-Router A should know that Router B is an alternative path. Router B is a Loop-Free Alternate for destination 10.1.1.0/24

OSPF LFA
Diagram: OSPF LFA.

 

This is not done with any tunneling, and the backup route needs to exist for it to be used by the RIB. If the second path doesn’t exist in the first place, the OSPF process cannot install a Loop-Free Alternative. The LFA process does not create backup routes if they don’t already exist. An LFA is simply an alternative loop-free route calculated at any network router.

A drawback of LFA is that it cannot work in all topologies. This is most notable in RING topologies. The answer is to tunnel and to get the traffic past the point where it will loop. This effectively makes the RING topology a MESH topology. For example, the diagram below recognizes that we must tunnel traffic from A to C. The tunnel type doesn’t matter – it could be a GRE tunnel, an MPLS tunnel, an IP-in-IP tunnel, or just about any other encapsulation.

 

In this network:

-Router A’s best path through E
-Routers C’s best path is through D
-Router A must forward traffic directly to C to prevent packets from looping back.

Remote LFA
Diagram: Remote LFA.

 

In the preceding example, we will look at “Remote LFA,” which leverages an MPLS network and Label Distribution Protocol ( LDP ) for label distribution. If you use Traffic Engineering ( TE ), it’s called “TE Fast ReRoute” and not “Remote LFA.” There is also a hybrid model combining Remote LFA and TE Fast ReRoute, and is used only when the above cannot work efficiently due to a complex topology or corner case scenario.

Remote LFAs extend the LFA space to “tunneled neighbors”.

– Router A runs a constrained SPF and finds C is a valid LFA

– Since C is not directly connected, Router A must tunnel to C

a) Router A uses LDP to configure an MPLS path to C

b) Installs this alternate path as an LFA in the CEF table

– If the A->E link fails.

a) Router A begins forwarding traffic along the LDP path

The total time for convergence usually takes 10ms.

Remote LFA has some topology constraints. For example, they cannot be calculated across a flooding domain boundary, i.e., an ABR in OSPF or L1/L2 boundary is IS-IS. However, they work in about 80% of all possible topologies and 90% of production topologies.

 

BGP Enhancements

BGP is a scalable distance vector protocol that runs on top of TCP. It uses a path algorithm to determine the best path to install in the IP routing table and for IP forwarding.

 

Recap BGP route advertisement:

  • RR client can send to a RR client.
  • RR client can send to a non RR client.
  • A non-RR client cannot send to a non-RR client.

One drawback to the default BGP behavior is that it only advertises the best route. When a BGP Route Reflector receives multiple paths to the same destination, it will advertise only one of those routes to its neighbors.

This can limit the visibility in the network and affect the best path selection used for hot potato routing when you want traffic to leave your AS as quickly as possible. In addition, all paths to exit an AS are not advertised to all peers, basically hiding ( not advertising ) some paths to exit the network.

The diagram below displays default BGP behavior; the RR receives two routes from PE2 and PE3 about destination Z; due to the BGP best path mechanism, it only advertises one of those paths to PE1. 

Route Reflector - Default
Diagram: Route Reflector – Default.

 

In certain designs, you could advertise the destination CE with different Route Distinguishers (RDs), creating two instances for the same destination prefix. This would allow the RR to send two paths to PE.

 

Diverse BGP path distribution

Another new feature is diverse BGP Path distribution, where you can create a shadow BGP session to the RR. It is easy to deploy, and the diverse iBGP session will announce the 2nd best path. Shadow BGP sessions are especially useful in virtualized deployments, where you can create another BGP session to a VM acting as a Route-Reflector. The VM can then be scaled out in a virtualized environment creating numerous BGP sessions. You are allowing the advertisements of multiple paths for each destination prefix.

Route Reflector - Shadow Sessions
Diagram: Route Reflector – Shadow Sessions.

 

BGP Add-path 

A special extension to BGP known as “Add Paths” allows BGP speakers to propagate and accept multiple paths for the same prefix. The BGP Add-Path feature will signal diverse paths, so you don’t need to create shadow BGP sessions. There is a requirement that all Add-Path receiver BGP routers must support the Add-Path capability.

There are two flavors of the Add-Path capability, Add-n-path, and Add-all-path. The “Add-n-path” will add 2 or 3 paths depending on the IOS version. With “Add-all-path,” the route reflector will do the primary best path computation (only on the first path) and then send all paths to the BR/PE. This is useful for large ECMP load balancing, where you need multiple existing paths for hot potato routing.

BGP Add Path
Diagram: BGP Add Path

 

Source packet routing

Another interesting draft the IETF is working on is Source Packet Routing ( spring ). Source Packet Routing is the ability of a node to specify a forwarding path. As the packet arrives in the network, the edge device looks at the application, determines what it needs, and predicts its path throughout the network. Segment routing leverages the MPLS data plane, i.e., push, swap, and pop controls, without needing LDP or RSVP-TE. This avoids millions of labels in the LDP database or TE LSPs in the networks.

 

Application Controls - Network DeliversDiagram: Application Controls – Network Delivers 

The complexity and state are now isolated to the network’s edges, and the middle nodes are only swapping labels. The source routing is based on the notion of a 32-bit segment that can represent any instructions, such as service, context, or IGP-based forwarding construct. This results in an ordered chain of topological and service instructions where the ingress node pushes the segment list on the packet.

 

Prefix Hijacking in BGP

BGP hijacking revolves around locating an ISP that is not filtering advertisements, or its misconfiguration makes it susceptible to a man-in-the-middle attack. Once located, an attacker can advertise any prefix they want, causing some or all traffic to be diverted from the real source towards the attacker.

In February 2008, a large portion of YouTube’s address space was redirected to Pakistan when the Pakistan Telecommunication Authority ( PTA ) decided to restrict access to YouTube.com inside the country but accidentally blackholed the route in the global BGP table.

These events and others have led the Secure-Inter Domain Routing Group ( SIDR ) to address the following two vulnerabilities in BGP:

-Is the AS authorized to originate an IP prefix?

-Is the AS-Path represented in the route the same as the path through which the NLRI traveled?

This lockdown of BGP has three solution components:

 

RPKI Infrastructure Offline repository of verifiable secure objects based on public-key cryptography
Follows resources (IPv4/v6 + ASN) allocation hierarchy to provide “right of use”
BGP Secure Origin AS You only validate the Origin AS of a BGP UPDATE
Solves most frequent incidents (*)
No changes to BGP nor the router’s hardware impact
Standardization is almost finished and running code
BGP PATH Validation BGPSEC proposal under development at IETF
Requires forward signing AS-PATH attribute
Changes in BGP and possible routers

The roll-out and implementation should be gradual and create islands of trust worldwide. These islands of trust will eventually interconnect together, making BGP more secure.

The table below displays the RPKI Deployment State;

RIR Total Valid Invalid Unknown Accuracy RPKI Adoption Rate
AFRINIC 100% .44% .42% 99.14% 51.49% .86%
APNIC 100% .22% .24% 99.5% 48.32% .46%
ARIN 100% .4% .14% 99.46% 74.65% .54%
LACNIC 100% 17.84% 2.01% 80.15% 89.87% 19.85%
RIPE NCC 100% 6.7% 0.59% 92.71% 91.92% 7.29%

Cloud Enhancements – The Intercloud

Today’s clouds have crossed well beyond the initial hype, and applications are now offered as on-demand services ( anything-as-a-service [XaaS] ). These services are making significant cost savings, and the cloud transition is shaping up to be as powerful as the previous one – the Internet. The Intercloud and the Internet of Things are the two new big clouds of the future.

Currently, the cloud market is driven by two spaces – the public cloud ( off-premise ) and the private cloud (on-premise). The intercloud takes the concept of cloud much further and attempts to connect multiple public clouds. A single application that could integrate services and workloads from ten or more clouds would create opportunities and potentially alter the cloud market landscape significantly. Hence, it is important to know and understand the need for cloud migration and its related problems.

We are already beginning to see signs of this in the current market. Various applications, such as Spotify and Google Maps, authenticate unregistered users with their Facebook credentials. Another use case is a cloud IaaS provider could divert incoming workload to another provider if it doesn’t have the resources to serve the incoming requests, essentially cloud bursting from provider to provider. It would also make economic sense to move workload and services between cloud providers based on cooling costs ( follow the moon ). Or maybe dynamically move workloads between providers, so they are closest to the active user base ( follow the sun )

The following diagram displays a Dynamic Workload Migration between two Cloud companies.

 

Intercloud
Diagram: Intercloud.

 

A: Cloud 1 finds Cloud 2 -Naming, Presence
B: Cloud 1 Trusts Cloud 2 -Certificates, Trustsec
C: Both Cloud 1 and 2 negotiate -Security, Policy
D: Cloud 1 sets up Cloud 2 -Placement, Deployment
E: Cloud 1 sends to Cloud 2 -VM runs in cloud-Addressing, configurations

The concept of Intercloud was difficult to achieve with the previous version of vSphere based on the restriction of latency for VMotion to operate efficiently. Now vSphere v6 can tolerate 100 msec of RTT.

InterCloud is still a conceptual framework, and the following questions must be addressed before it can be moved from concept to production.

1) Intercloud security

2) Intercloud SLA management

3) Interoperability across cloud providers.

 

Cisco’s One Platform Kit (onePK)

The One Platform Kit is Cisco’s answer to Software Defined Networking. It aims to provide simplicity and agility to a programmatic network. It’s a set of APIs driven by programming languages, such as C and Java, that are used to program the network. We currently have existing ways to program the network with EEM applets but lack an automation tool that can program multiple devices simultaneously. It’s the same with Performance Routing ( PfR ). PfR can program and traffic engineer the network by remotely changing metrics, but the decisions are still local and not controller-based.

 

Traffic engineering

One useful element of Cisco’s One Platform Kit is its ability to perform “Off box” traffic engineering, i.e., the computation is made outside the local routing device. It allows you to create route paths throughout the network without relying on default routing protocol behavior. For example, the cost is the default metric for route selection for equal-length routes in OSPF. This cannot be changed, which makes the routing decisions very static. In addition, Cisco’s One Platform Kit (onePK) allows you to calculate routes using different variables you set, giving you complete path control.

 

ip routing

Green data center with eco friendly electricity usage tiny person concept. Database server technology for file storage hosting with ecological and carbon neutral power source vector illustration.

Data Center Design with Active Active design

by Blog

Network Insight · High Availability Architecture

Active-Active Data Center Design: Building for Continuous Availability

What happens when one data center can no longer serve its workloads? An active-active architecture aims to keep multiple sites operational at the same time, distributing traffic and capacity while preparing services to withstand component, network and site failures.

Modern applications rarely depend on a single network, server or storage system. They operate across interconnected infrastructure, distributed services and, increasingly, multiple locations. Active-active design brings these components together into an architecture where more than one site can actively serve workloads. But operating two sites simultaneously is not simply a matter of duplicating hardware. Routing, application behavior, data consistency, traffic distribution and failure detection must work together.

Data Center A

Primary active site in this illustration

Applications Compute Storage
Serving workloads

Data Center B

Second active site in this illustration

Applications Compute Storage
Serving workloads

01 · Connectivity

Route around failures

Use diverse network paths and appropriate routing policies to preserve reachability when links or devices fail.

02 · Consistency

Keep application state safe

Design replication, quorum and application behavior around the consistency requirements of the service.

03 · Recovery

Prove service continuity

Test site isolation, traffic shifts and recovery behavior instead of assuming that redundancy guarantees availability.

i

The essential distinction: both sites can be active without every application being able to survive every failure. True service resilience depends on the complete design—including data consistency, health detection, routing, capacity and application recovery behavior.

Multi-site networking BGP and DCI Data consistency High availability Failure engineering
Section 01 · Foundations

Understanding Active-Active Data Center Design

Explore how two or more data center sites can serve workloads concurrently, how this differs from active-passive resilience, and why application behaviour, dependency design and failure handling matter as much as network redundancy.

Active vs. passiveAvailabilityFailure domains
Section 01 · Architecture Fundamentals

Understanding Active-Active Data Center Design

Active-active design allows two or more data center sites to serve production workloads concurrently. The objective is to use distributed capacity while maintaining service continuity when a component, network path or site becomes unavailable.

Traditional active-passive designs generally keep one site serving traffic while another is prepared to take over during a failure. In an active-active design, both sites perform useful production work during normal operation. Traffic may be distributed through global load balancing, application-aware routing, anycast services or other mechanisms appropriate to the application and network architecture.

This can improve resource utilisation and provide additional capacity options. It also introduces design questions that cannot be solved by routing alone: where application state lives, how data is replicated, how clients reach healthy services, and what happens when the sites lose contact with one another.

Three principles that shape the design

01

Concurrent workload

Both sites contribute to normal service delivery. Capacity planning must account for traffic distribution and whether surviving infrastructure can absorb additional demand after a failure.

02

Consistent service behaviour

Applications must handle sessions, shared state, data replication and concurrent updates according to their consistency and availability requirements.

03

Designed failure response

Health checks, routing policies, load balancers and application recovery logic must work together to direct clients towards services that can actually respond.

Important distinction: two active sites do not automatically provide application resilience. If both sites depend on the same database, identity service, WAN path or control-plane component, that shared dependency may still become a single point of failure.

Questions to answer before choosing active-active

  • Can the application safely run across multiple sites at the same time?
  • How much data loss, if any, is acceptable following a site failure?
  • What recovery time is required, and how will client traffic move to healthy services?
  • Can one surviving site handle the required workload during a major failure?
  • How will split-brain conditions, network partitions and inconsistent state be detected and handled?
Engineering takeaway

Active-active is an end-to-end service architecture, not simply a pair of interconnected data centers. Reliable outcomes depend on coordinated network design, application behaviour, data consistency, capacity planning and tested recovery procedures.

Interactive Lab 01 · Architecture explorer

What Happens When a Data Center Fails?

Switch the deployment model or simulate a site failure. Observe the illustrative traffic paths, site states and engineering implications. This is a conceptual model, not a live network simulator.

Choose a scenario
Two-site data center architecture Two data center sites connect to a shared user traffic entry point. The selected scenario changes the illustrative traffic paths and site status. Users / Traffic Entry DNS · GSLB · Load balancing Data Center A Data Center B Serving workloads Serving workloads Application stack A ACTIVE Application stack B ACTIVE Inter-site connectivity / replication Both sites serve traffic concurrently
Data Center A
Active · serving traffic
Data Center B
Active · serving traffic
SCENARIO: ACTIVE-ACTIVE

Both sites serve production traffic

Workloads at both sites are available to serve requests. Traffic distribution depends on application placement, routing, load balancing and health checks. Shared dependencies and data consistency still need deliberate design.

Engineering checks

  • Confirm application and data consistency.
  • Verify each site's independent capacity.
  • Test routing and service health during failure.
Illustrative topology · No live devices, routing protocols or production traffic are controlled.
Section 02 · Routing architecture

Multi-Site Network Architecture and BGP

Follow traffic between data centers and external networks. Understand how BGP exchanges reachability, how routing policy and multipath influence path selection, and how engineers avoid asymmetric traffic, route leaks and unexpected failover behaviour.

DCI designBGP policyTraffic engineering
Section 02 · Network Architecture

Multi-Site Network Architecture and BGP

An active-active design needs more than two functioning data centers. It needs predictable routing, resilient inter-site connectivity and a clear policy for how users and applications reach services when paths or entire sites fail.

The network provides the connectivity between data centers, users, applications and external services. In a multi-site architecture, Data Center Interconnect (DCI) links connect the sites, while routing and traffic-management systems determine which paths are available and where traffic should be sent.

Border Gateway Protocol (BGP) is commonly used to exchange reachability information between sites, WAN networks and external networks. Its policy controls can influence path selection and traffic distribution. However, BGP does not by itself verify that an application is healthy or that its data is consistent.

The three network design layers

Layer 01

Data Center Interconnect

DCI may use routed IP connectivity, an optical transport service or an overlay architecture. Choose bandwidth, latency and path diversity according to application traffic and replication requirements—not simply the number of physical links.

Layer 02

Routing and reachability

BGP can advertise site prefixes and apply routing policy. Multipath forwarding may use multiple eligible routes when supported and correctly configured. Route selection and forwarding behaviour must be validated for the actual platform and topology.

Layer 03

Service traffic steering

DNS-based global load balancing, anycast, application load balancers or other steering mechanisms can direct clients towards available services. Health checks must reflect genuine service readiness, not just network reachability.

BGP: control the path, then validate the outcome

A common design advertises service or site prefixes from more than one location. The routing policy determines which advertisements are accepted and how paths are preferred. Depending on the design, engineers may use attributes such as LOCAL_PREF within an autonomous system, AS_PATH for inter-AS policy, or multipath features to support selected equal-cost paths.

The important distinction is between route availability and service availability. A router may have a valid route to a site whose application is unhealthy. Conversely, an application may be healthy while a routing or traffic-steering policy prevents clients from reaching it. These systems need coordinated health signals and carefully tested failure behaviour.

Design warning — avoid accidental traffic tromboning. If a client reaches a service in one site but the application retrieves data or calls a dependency in the other site, traffic may cross the DCI unnecessarily. This can increase latency, consume inter-site bandwidth and complicate failure handling. Keep traffic local where practical, while preserving the required consistency and availability guarantees.

What should happen during a failure?

  • DCI link failure: traffic should use a surviving path where one exists, with convergence and capacity validated under load.
  • Border router failure: alternate routing paths should remain available without creating a forwarding loop or black hole.
  • Site failure: route withdrawal and service-health mechanisms should steer new traffic away from unavailable services.
  • Application failure: health checks should stop advertising or selecting an endpoint that cannot deliver the required service.
  • Network partition: routing and application policies must avoid unsafe assumptions about the remote site's state or data ownership.

Architecture checks before deployment

  • Identify shared physical and logical failure domains across DCI paths.
  • Document prefix advertisements, routing policy, route filtering and intended path preferences.
  • Check convergence behaviour and whether surviving links can carry the expected load.
  • Validate inbound and outbound traffic steering independently; they may follow different paths.
  • Test application health checks alongside routing changes and site-isolation scenarios.
Engineering takeaway

BGP and DCI establish and control network reachability; they do not guarantee that an application is usable. A resilient active-active design coordinates routing, traffic steering, application health, capacity and data consistency—and verifies the result through controlled failure tests.

Interactive Lab 02 · Multi-site routing

Trace the Route Between Data Centers

Explore how inter-site connectivity, BGP reachability and routing policy influence traffic. Simulate a DCI link failure or route withdrawal, then select a router or network to inspect its role.

Select a routing scenario
Illustrative multi-site BGP routing topology Two data centers with border routers connect through an inter-site link. An external network advertises a prefix. Select a node to inspect its role. SITE A · AS 65001 SITE B · AS 65001 Application A 10.10.10.0/24 Border Router A BGP / route policy Border Router B BGP / route policy Application B Service destination External Network AS 65002 Prefix advertisement DCI · inter-site path eBGP iBGP / local routing iBGP / local routing Illustrative route: 198.51.100.0/24 Conceptual diagram · BGP behaviour depends on topology and policy
Illustrative preferred path Illustrative alternate path Failed or withdrawn path
STATE: NORMAL ROUTING

Reachability is available through the configured paths

The diagram illustrates an available inter-site path and route advertisement. Actual forwarding depends on the installed route, routing policy, next-hop reachability and data-plane health.

Select a network component

Select either border router, an application network or the external AS in the diagram to explore its role in route propagation and traffic forwarding.

DCI statusAvailable
Illustrative routeAdvertised
Engineering focusValidate policy
ItemIllustrative stateWhat to verify
Inter-site pathAvailableIndependent physical paths, capacity and forwarding health.
Route advertisementPresentAccepted prefixes, next hop, policy and route installation.
Service reachabilityCheck requiredEnd-to-end application health, not only control-plane state.
Educational model only · No live BGP sessions or production routes are changed.
Section 03 · Overlays and mobility

Layer 2 Extension, Overlays and Workload Mobility

Compare stretched Layer 2 networks with routed Layer 3 designs, and examine how VXLAN/EVPN overlays can support multi-site connectivity. Consider workload mobility, broadcast scope, failure propagation and the risk of traffic tromboning before extending networks between sites.

VXLAN / EVPNLayer 2 vs. Layer 3Workload mobility
Section 03 · Layer 2 and Overlay Design

Layer 2 Extension, Overlays and Workload Mobility

Workload mobility across data centers can simplify some application migrations and recovery scenarios. The challenge is deciding how much network continuity is actually required—and whether extending a Layer 2 domain is the right way to achieve it.

In a multi-site environment, applications may need to move between sites for maintenance, capacity balancing, disaster recovery or infrastructure changes. Some workloads can move while retaining their IP addresses; others can reconnect through a new endpoint or service address. The application’s requirements should determine the network design, rather than assuming that every workload needs a stretched subnet.

A Layer 2 extension carries an Ethernet broadcast domain across multiple locations. This can preserve subnet membership and support certain migration workflows, but it also extends the domain’s failure and operational characteristics. A Layer 3 design keeps subnets local to each site and uses routing to connect them, often reducing the scope of Layer 2 failure propagation.

Approach A

Stretched Layer 2

The same VLAN or subnet is available across sites, subject to the chosen extension technology and design constraints.

  • Can support address-preserving workload mobility.
  • May carry broadcast and unknown-unicast traffic across the DCI.
  • Requires careful control of loops, failure domains and inter-site latency.
  • Can make partial failures harder to diagnose.
Approach B

Routed Layer 3

Each site uses its own routed subnets, and connectivity between sites is provided through routing and application-level service discovery or traffic steering.

  • Provides clearer Layer 3 failure boundaries.
  • Supports route summarisation and explicit traffic policy.
  • May require application, DNS or load-balancer changes during migration.
  • Often simplifies fault isolation and scaling.

VXLAN and EVPN: separating the overlay from the underlay

VXLAN (Virtual Extensible LAN) encapsulates Layer 2 frames in UDP packets so that virtual networks can be carried across an IP underlay. VXLAN Tunnel Endpoints (VTEPs) perform encapsulation and decapsulation. EVPN (Ethernet VPN), commonly used as a control plane for VXLAN fabrics, distributes MAC and IP reachability information and supports policy-driven tenant connectivity.

This separation allows an IP-routed fabric to carry multiple logical networks without requiring the physical underlay itself to be one large Layer 2 domain. EVPN can support distributed gateway designs and multi-homing, depending on the implementation. It does not, however, remove the need to design inter-site routing, failure handling and application consistency deliberately.

Where do OTV, FabricPath and LISP fit?

Legacy DCI

OTV

Overlay Transport Virtualization was designed to extend selected Layer 2 connectivity between data centers while limiting some traditional Layer 2 extension risks. Treat it as a platform- and lifecycle-dependent technology, not a universal modern default.

Fabric technology

FabricPath

Cisco FabricPath introduced a Layer 2 fabric approach using a fabric control plane rather than relying on conventional spanning tree for all forwarding decisions. It is mainly relevant when understanding existing or legacy environments.

Identifier / locator

LISP

Locator/ID Separation Protocol separates endpoint identity from routing location. It can support mobility and mapping use cases in appropriate architectures, but it is not interchangeable with VXLAN or EVPN.

Design principle: do not extend Layer 2 simply because an application currently uses a fixed IP address. First check whether the application can tolerate a different subnet, use a stable service endpoint, or reconnect through DNS or a load balancer. A routed design may meet the requirement with a smaller failure domain.

Workload mobility and traffic tromboning

Preserving a workload’s IP address does not guarantee that its traffic path remains optimal. After a workload moves, clients or dependent services may still reach it through a remote site. This can create traffic tromboning: packets travel across the DCI to reach a service and may cross back again to reach another dependency.

The result can be increased latency, unnecessary bandwidth consumption and a more complicated failure path. Mobility planning should therefore include endpoint discovery, routing updates, load-balancer behaviour, dependency locality and the time needed for existing sessions to recover.

Questions to resolve before extending a network

  • Does the workload genuinely require the same subnet at both sites?
  • What are the latency, bandwidth and MTU requirements across the DCI?
  • How are MAC/IP reachability and gateway behaviour handled after a move or failure?
  • Can a Layer 2 loop, broadcast event or control-plane issue affect both sites?
  • Will traffic remain local to the active workload and its dependencies?
  • What happens to established sessions when the workload or site becomes unavailable?
Engineering takeaway

VXLAN and EVPN can provide scalable overlay connectivity, while routed designs can keep site boundaries clearer. Choose Layer 2 extension only when the application and operational requirements justify it, and test mobility, failure isolation and traffic locality before relying on it for resilience.

Interactive design lab · 03

Layer 2 Extension, Overlays and Workload Mobility

Compare three ways to connect data-centre sites. Change the design, simulate an inter-site failure, and examine what happens to addressing, mobility and the fault domain.

1. Choose a connectivity design
2. Test the inter-site connection
Conceptual topology Shared Layer 2 segment
Two-site data-centre connectivity diagram Two data-centre sites with workloads and network gateways connected by an inter-site path. The selected mode changes the path labels and the illustrative failure state. DATA CENTRE A Primary site · example workload DATA CENTRE B Secondary site · example workload Application workload A VLAN 120 · 10.20.120.0/24 Application workload B VLAN 120 · 10.20.120.0/24 Site A gateway Local forwarding Site B gateway Local forwarding L2 extension One broadcast domain Inter-site path operational
✓
Shared segment is available

Both sites use the same example subnet. This can support address continuity, but it also extends Layer 2 dependencies across the sites.

Addressing Same subnet at both sites
Mobility approach Layer 2 continuity
Failure consideration Broadcast and loop control

Engineering trade-off

A stretched VLAN can make subnet continuity easier, but the design must deliberately control loop prevention, broadcast scope, gateway behaviour and inter-site failure handling.

  • Define the Layer 2 extension boundary
  • Test behaviour during link and site failure
  • Validate gateway redundancy and convergence
  • Confirm application recovery requirements
Educational topology only — not a live network simulator. VXLAN is an encapsulation technology; EVPN can provide a BGP-based control plane. Actual mobility, redundancy and failure behaviour depend on the chosen platform, configuration and application design.
Section 04 · Application resilience

Application, Storage and State Consistency

Network redundancy cannot guarantee application availability on its own. Examine stateless and stateful services, session continuity, synchronous and asynchronous replication, quorum, split-brain risk, and the trade-offs between recovery time and potential data loss.

Application stateReplicationRPO / RTO
Section 04 · Application and Data Resilience

Application, Storage and State Consistency

A network can keep routes available while an application remains unable to serve requests. Active-active resilience therefore depends on how application instances, user sessions and persistent data behave across sites—not just on redundant links and routers.

In an active-active environment, application instances in different data centers may process requests concurrently. That arrangement works well only when the application can manage shared state, concurrent operations and failures consistently. A stateless service is often easier to distribute because each request can be handled by any healthy instance, provided its dependencies are also available.

Stateful applications introduce more complexity. They may depend on databases, distributed caches, message queues, session stores or file systems. Each dependency needs an explicit availability and consistency strategy. Replicating the application tier alone does not make the whole service resilient if its database or identity provider remains a single point of failure.

Synchronous versus asynchronous replication

Replication determines how updates made at one site become available at another. The key trade-off is between acknowledgement latency, consistency guarantees and the amount of data that may be lost during a failure. These characteristics vary by platform and configuration; the terms alone do not guarantee a particular recovery outcome.

Replication model A

Synchronous replication

A write is acknowledged only after the configured synchronous durability or commit conditions have been met.

  • Can reduce the recovery point exposure for acknowledged writes.
  • Inter-site latency can affect write response times.
  • A link or site failure may affect write availability, depending on quorum and product design.
  • Requires careful planning for network partitions and failover.
Replication model B

Asynchronous replication

A write can be acknowledged before the remote replica has received or committed the update.

  • Can reduce the impact of remote-site latency on write acknowledgement.
  • Replication lag may leave the remote site temporarily behind.
  • A sudden failure can result in loss of updates not yet replicated.
  • Recovery procedures must account for lag and data reconciliation.

Important: synchronous replication does not automatically mean zero data loss in every failure scenario, and asynchronous replication does not always imply unacceptable loss. The real outcome depends on commit semantics, quorum, durable storage, topology and the failure being considered. Define the required Recovery Point Objective (RPO) and Recovery Time Objective (RTO), then validate them against the actual platform.

Three application-level challenges

Challenge 01

Session continuity

If a client moves between sites, its session may depend on local memory, a shared session store, a signed token or another mechanism. Design for reconnects and retries instead of assuming a transport path change preserves every session.

Challenge 02

Concurrent updates

Two active instances may attempt to modify the same logical record. Transactions, conflict resolution, idempotency and application-specific consistency rules must prevent duplicated or conflicting operations.

Challenge 03

Dependency availability

An application instance may be healthy while its database, authentication service, queue or external API is not. Service health should reflect the dependencies required to complete real user operations.

Quorum, network partitions and split-brain

A network partition can leave two sites running while preventing them from communicating. If both sides independently accept conflicting writes or claim ownership of the same resource, the system may enter a split-brain condition. Restoring connectivity does not necessarily reconcile the resulting state safely.

Distributed systems often use quorum, leases, fencing tokens, leader election or other coordination mechanisms to decide which operations are permitted during a partition. A two-site deployment may need a third failure-independent voting participant or witness, depending on the platform. A witness can help establish a decision, but it does not replace data replication or application-level recovery design.

Network partition is not the same as site failure. If the DCI fails, both sites may still be running and serving local clients. The architecture must decide whether each site can safely continue, whether one side must stop writes, and how service ownership is re-established when communication returns.

Questions to answer before production

  • Can application instances safely process requests concurrently in both sites?
  • Where is session state stored, and what happens when a client is redirected?
  • What RPO and RTO are required for each critical data store?
  • How are quorum, ownership and split-brain conditions handled during a partition?
  • Can the surviving site sustain the required transaction rate and connection load?
  • How will replication lag, failed writes, stale reads and recovery conflicts be detected?
  • Have failover and failback been tested with real application transactions, not just ping tests?
Engineering takeaway

Active-active application resilience requires an explicit strategy for state, replication, coordination and recovery. Network redundancy helps services communicate, but only the combined behaviour of the application, data platform and network determines whether service can continue safely during a failure.

Interactive design lab · 04

Application, Storage and State Consistency

A network can remain connected while an application becomes unsafe or unavailable. Explore replication modes, simulate an inter-site partition, and examine why two active sites need a deliberate write-ownership strategy.

1. Choose a data replication model
2. Choose a scenario
Application and data flow Synchronous data replication
Two-site application and data replication model Two data-centre sites contain application instances and data services. A solid path represents client and application connectivity, while a dashed path represents replication or inter-site coordination. DATA CENTRE A Serving requests · replica A DATA CENTRE B Serving requests · replica B Application instance A Accepting requests Application instance B Accepting requests Data replica A Commit acknowledgement Data replica B Remote copy Commit coordination Remote acknowledgement Inter-site coordination available
✓
Replication is operating normally

In this conceptual synchronous model, a write waits for the configured remote acknowledgement before it is confirmed. Real guarantees depend on the data platform and commit policy.

Write behaviour Waits for required acknowledgement
Data risk Depends on commit and quorum rules
Recovery priority Validate quorum and ownership

Engineering checks

A healthy network path does not prove that a transaction is durable or that both replicas agree. Check the application's consistency contract and the data platform's failure behaviour.

  • Confirm what a successful commit guarantees
  • Define quorum and write ownership
  • Measure replication delay and transaction errors
  • Test failover, fencing and safe recovery
Educational model only — no real transactions or replication occur. The diagram illustrates design trade-offs, not a guarantee that a particular database or storage platform behaves this way. Synchronous replication does not automatically prevent data loss in every failure; asynchronous replication may have a non-zero recovery point.
Section 05 · Operations and recovery

Failure Scenarios, Observability and Recovery

Move beyond theoretical redundancy and test how the architecture behaves under real failure conditions. Trace link loss, BGP changes, application dependency failures, replication lag and site isolation using telemetry, incident procedures and measured recovery objectives.

Failure testingTelemetryFailover and failback
Section 05 · Observability and Recovery

Failure Scenarios, Observability and Recovery

Active-active architecture must be designed around failure, not just normal operation. Reliable recovery depends on detecting what has failed, identifying the affected services, understanding the data state and directing traffic towards capacity that can safely serve it.

Redundant links and duplicated infrastructure reduce some risks, but they do not eliminate failures. A site can remain reachable while its applications are unhealthy; a routing session can remain established while packets are being dropped; and a database replica can be online while lagging behind the active data set. Effective operations require end-to-end observability across network, application and data layers.

Detect

Identify the real failure

Use telemetry and health checks to distinguish a failed link, an unreachable site, a degraded application and an unhealthy data dependency.

Decide

Choose a safe response

Determine whether traffic should move, whether writes can continue, and whether the surviving site has enough capacity to take additional work.

Validate

Prove service recovery

Confirm that users can complete meaningful transactions, not merely that interfaces are up or routing adjacencies have recovered.

Five failure scenarios to test

Inter-data-center link failure

One DCI path fails while both sites remain operational. Check route convergence, alternate-path capacity, packet loss and whether replication or application dependencies are affected.

Validate: path failover · latency · loss · capacity

Border router or routing failure

A border device fails, a route is withdrawn or a policy change directs traffic incorrectly. Verify that alternate routes are usable and that recovery does not create a forwarding loop or black hole.

Validate: BGP state · route changes · forwarding path

Application or dependency failure

Network connectivity remains available, but an application instance, database or identity service cannot complete requests. Check whether health signals remove the affected endpoint from service selection.

Validate: synthetic transactions · dependency health

Replication lag or data inconsistency

The remote data copy falls behind or a replication process stops. Establish whether the secondary site is safe to promote, what data could be missing and how consistency will be verified before normal service resumes.

Validate: replication lag · RPO · recovery procedure

Complete site loss or network partition

A site becomes unavailable, or both sites remain running but lose communication. Confirm the failover decision, capacity at the surviving site, data ownership and safeguards against split-brain.

Validate: site isolation · quorum · service continuity

A practical incident investigation sequence

During an incident, investigate from the observed user impact towards the underlying dependency. Correlating timestamps across routing events, interface counters, application logs and replication metrics helps distinguish the initiating failure from symptoms that appear later.

From alert to verified recovery Operational workflow
1 Scope impact

Identify affected users, services, sites and transactions.

2 Correlate signals

Compare network, application and data telemetry.

3 Apply safe recovery

Follow tested routing, failover and data procedures.

4 Verify service

Test transactions, data integrity and performance.

This is a general investigation pattern. Follow your incident procedures and service-specific recovery runbooks when making production changes.

What should you monitor?

Layer Useful signals What they help reveal
Network Interface errors, packet loss, latency, BGP changes, path utilisation Transport degradation, route changes and capacity pressure
Application Request success rate, latency, error rate, synthetic transactions Whether users can actually complete operations
Data Replication lag, commit failures, consistency checks, storage health Whether the remote site has a safe and sufficiently current data set
Platform CPU, memory, connection pools, queue depth, storage throughput Whether the surviving site can absorb additional workload
Recovery Failover duration, RTO/RPO achievement, recovery test results Whether the design meets its agreed recovery objectives

Do not confuse an infrastructure health check with a service health check. An interface can be up, a BGP session can be established and a server can respond to ping while a real transaction fails. Use application-aware probes and representative synthetic transactions wherever practical.

Failover, failback and recovery objectives

Recovery Time Objective (RTO) defines the target time to restore a service after disruption. Recovery Point Objective (RPO) defines the acceptable data-loss window, expressed as time or an equivalent recovery-point requirement. Different applications and data stores may need different objectives; a single pair of numbers should not automatically be applied to every service.

Failover is only half of the recovery story. Failback—returning service to its intended operating arrangement—can introduce another period of risk. Before returning workloads, confirm data consistency, replication health, routing policy, capacity and the correct ownership of stateful services. Avoid automatic failback if it could trigger repeated site switching or disrupt active sessions.

Operational readiness checklist

  • Document the expected behaviour for link, device, application, data and full-site failures.
  • Ensure alerts identify the affected service and provide enough context for investigation.
  • Test routing convergence and verify actual packet forwarding after route changes.
  • Confirm that the surviving site has enough compute, storage, network and application capacity.
  • Define who can authorise failover, data promotion and failback during an incident.
  • Record measured recovery times and data outcomes during controlled exercises.
  • Review failures and near misses to improve runbooks, monitoring and architecture.
Engineering takeaway

Resilience must be measured through observed service outcomes. Combine network telemetry, application health, data replication metrics and tested recovery procedures to determine whether the active-active architecture can withstand the failures it was designed to handle.

Interactive design lab · 05

Failure Scenarios, Observability and Recovery

Simulate a router failure, an inter-site partition or a complete site outage. Observe the illustrative network state, monitoring signals and recovery checks that matter before declaring an active-active service healthy.

1. Select a failure scenario
Illustrative topology and alternate path All paths operational
Active-active data-centre failure scenario Two data-centre sites connect through border routers and an inter-site link. A separate illustrative alternate path is shown. Select a scenario to change the topology and monitoring indicators. DATA CENTRE A Site healthy DATA CENTRE B Site healthy Application service A Serving requests Application service B Serving requests Border router A BGP established Border router B BGP established Inter-site path Illustrative alternate path
✓
Both sites are operational

The conceptual topology shows active services at both sites. Monitoring should confirm end-to-end transaction success, route health, replication status and available capacity.

Network reachability Primary and inter-site paths available
Application service Both sites serving
Recovery decision Monitor and validate health

Observability panel

Routing adjacencyHealthy
End-to-end transactionsHealthy
Inter-site connectivityHealthy
Replication / coordinationHealthy

Recovery checklist

A green interface or established BGP session is only one signal. Correlate network telemetry with application transactions, data health and the capacity of the surviving site.

  • Confirm the failure and its scope
  • Validate the alternate forwarding path
  • Check application and data integrity
  • Verify recovery objectives are met
Educational simulation only. No live devices, BGP sessions or monitoring systems are contacted. The alternate path is illustrative; real failover depends on topology, routing policy, detection timers, capacity, application dependencies and the platform's recovery behaviour.
Section 06 · Hybrid architecture

Cloud Connectivity and Modern Design Choices

Explore how data centers connect to cloud networks using private connectivity, VPNs, dynamic routing, VPC peering and hub-based services such as Google Cloud Network Connectivity Center. Compare resilience, route control, security, latency and cost without assuming one cloud model fits every deployment.

Hybrid cloudVPC and BGPConnectivity choices
Active-Active Data Center Design · Section 06

Cloud Connectivity and Modern Design Choices

Active-active design does not stop at the data center perimeter. Applications may depend on private cloud services, public cloud workloads, shared identity systems, or data platforms in another environment. The challenge is to connect these resources without creating hidden dependencies, unnecessary latency, or new single points of failure.

1. Separate data center interconnect from hybrid cloud connectivity

A data center interconnect (DCI) connects separate data center sites. Hybrid cloud connectivity connects an enterprise network or data center to cloud networks and services. These designs can overlap, but they solve different problems and should be assessed independently.

For example, two active data centers may exchange application traffic over a dedicated DCI while each site also has its own connection to a cloud provider. Another design may route both sites through a shared cloud connectivity hub. The right choice depends on application placement, traffic patterns, security boundaries, and failure-domain requirements.

Illustrative hybrid connectivity model
Data Center A Active workloads Local routing and security
Private DCI or routed inter-site path
Data Center B Active workloads Independent failure domain
Enterprise / DCI edge Routing policy and controlled egress
Private connectivity or VPN
Cloud network VPCs, cloud workloads and services
This is a conceptual view, not a required physical topology. Connectivity can be direct, hub-and-spoke, or provider-mediated. Redundancy must be verified across the complete path, not just at the visible connection.

2. Choose the connectivity model for the workload

Cloud connectivity is not one universal design. Dedicated private links, provider-managed private connectivity, and IPsec VPNs offer different trade-offs in capacity, latency, encryption, deployment time, and cost.

01
Private connectivity

Often suited to predictable, sustained traffic and workloads with demanding bandwidth or latency requirements. Validate provider availability, port and circuit diversity, routing hand-offs, and the actual end-to-end service commitment.

02
Site-to-site VPN

IPsec VPN can provide encrypted connectivity over an IP underlay. High-availability VPN designs use redundant tunnels or endpoints, but the underlying internet paths and provider failure domains still matter.

03
Cloud routing

Cloud routers and BGP sessions can exchange routes dynamically where supported. Define which prefixes are advertised, accepted, preferred, or filtered, and test behaviour when a peer or path fails.

04
Hub-and-spoke connectivity

A cloud transit or connectivity hub can simplify connections between multiple networks. It also becomes an important policy and operational component that needs its own resilience and capacity planning.

3. Understand VPCs, peering and cloud connectivity hubs

In Google Cloud, a Virtual Private Cloud (VPC) provides a logical network boundary for cloud resources. VPC connectivity, Cloud VPN, Cloud Router, and Network Connectivity Center (NCC) address different parts of hybrid and multi-network design. Other providers have comparable concepts, but their capabilities and terminology differ.

Building block Design role Important consideration
VPC network Provides a logical network environment for cloud resources. Plan address ranges, segmentation, route policy, and access controls.
VPC peering Connects eligible VPC networks through a peering relationship. Do not assume peering provides transitive routing through another VPC.
Cloud VPN Provides IPsec-based connectivity between networks. Plan tunnel redundancy, routing, encryption requirements, and failover testing.
Cloud Router / BGP Supports dynamic route exchange for compatible connectivity services. Validate advertised prefixes, learned routes, route priorities, and withdrawal behaviour.
Network Connectivity Center Provides a hub-based framework for supported network connectivity patterns. Check supported resources, topology constraints, route propagation, and configuration requirements.
Do not confuse connectivity with end-to-end resilience

A cloud hub, a pair of VPN tunnels, or two physical circuits does not automatically make an application highly available. Shared carrier infrastructure, common provider endpoints, route-policy errors, cloud-region dependencies, DNS, identity, and application state can still create a common failure mode.

4. Design the routing and failure behaviour

Once connectivity is established, define what happens when a link, tunnel, router, data center, or cloud attachment becomes unavailable. Routing must support the intended failover behaviour without creating loops, accidental transit, or unintended access between security zones.

Route control

Advertise only required prefixes. Apply explicit import and export policy, avoid overlapping address spaces, and understand how route preference changes during failure.

Path diversity

Check whether redundant connections use independent devices, facilities, carriers, cloud endpoints, and regions where the availability objective requires it.

Security boundaries

Apply segmentation, firewall policy, identity controls, and inspection requirements to both primary and failover paths. A backup route should not bypass required controls.

Application dependencies

Validate DNS, authentication, secrets, databases, monitoring, and other shared services. Network reachability alone does not prove that the service can operate.

5. Compare the design options before committing

Select connectivity according to measured workload requirements and operational constraints—not simply because a particular product or topology is available.

Design question What to evaluate
Traffic profile Peak and sustained throughput, burst behaviour, packet size, and growth expectations.
Performance Latency, jitter, packet loss, regional distance, and application sensitivity.
Availability Independent paths, redundant endpoints, provider failure domains, and tested recovery behaviour.
Routing Address planning, BGP policy, route propagation, failover timing, and prevention of unintended transit.
Security Encryption, segmentation, inspection, identity, and compliance requirements.
Operations and cost Provisioning lead time, monitoring, support ownership, data transfer charges, and ongoing operational effort.

6. Validate the architecture with failure tests

Test the complete application path, including the network and the dependencies it uses. Record what users experience, how quickly the system detects a failure, which routes change, and whether recovery occurs within the required objective.

  • Disconnect one connectivity path and verify traffic moves as designed.
  • Withdraw a route and confirm that only intended prefixes change reachability.
  • Test a cloud attachment or VPN endpoint failure where the environment permits.
  • Verify that security policy remains enforced on the alternate path.
  • Measure application recovery, not just routing convergence.
  • Document ownership, alerts, escalation, and restoration procedures.
Engineering takeaway

Active-active data center design and hybrid cloud connectivity should be designed together where workloads depend on both—but they are not the same problem. Choose private connectivity, VPN, peering, or a hub-based model according to workload needs, then validate route policy, failure-domain independence, security, and application recovery end to end.

Design note: cloud networking products and capabilities change over time. Confirm current provider documentation, regional availability, quotas, and supported topology before implementation.
Interactive design explorer · Section 6

Cloud Connectivity and Modern Design Choices

Choose a connectivity pattern to explore how an on-premises network can connect to cloud networks. Compare the conceptual architecture, routing considerations and security decisions before selecting a design for a real environment.

01 / Select a connectivity pattern
02 / Architecture view Encrypted tunnel
Site-to-site VPN cloud connectivity An on-premises network connects through a VPN gateway to a cloud network. The tunnel is conceptual; actual routing and redundancy depend on configuration. ON-PREMISES Enterprise network Core router BGP / static Workloads Private LAN CLOUD Virtual network Gateway VPN endpoint Cloud apps Private subnets IPsec VPN tunnel Routes configured at both ends Tunnel redundancy depends on deployment

Conceptual diagram, not a provider-specific reference architecture. Redundant tunnels, gateways and route exchange must be designed and tested explicitly.

03 / Design assessment

Site-to-site VPN

Build an encrypted connection between the enterprise network and the cloud provider's VPN endpoint. The internet may carry the tunnel traffic, while routing controls which prefixes can traverse it.

Connection model Encrypted tunnel
Routing focus Prefixes and failover
Security focus IKE / IPsec policy
Resilience focus Multiple tunnels
Engineering checks
  • Confirm tunnel encryption, authentication and key-management requirements.
  • Validate route advertisements, allowed prefixes and return-path symmetry.
  • Test failure detection, alternate paths, throughput and maximum packet size.
Watch-outs
  • A tunnel being established does not prove application reachability.
  • One tunnel or gateway can remain a single point of failure.

High availability is an end-to-end property

Consider independent circuits, gateways, cloud regions or zones where supported, route convergence, capacity under failure, and the application's own dependency model. Two connections that share the same failure domain may not provide meaningful redundancy.

Routing and security are separate decisions

Route exchange determines reachability; firewall policy, segmentation, identity and encryption determine what is permitted and protected. Validate both directions and test the application path rather than relying on a successful routing adjacency alone.

Educational design explorer only — not a live cloud configuration tool. Provider capabilities, terminology, limits and route behaviour vary by service, region and configuration. Confirm the relevant provider documentation before implementation.
Network Insight · Article Summary

Active-Active Data Center Design: Key Takeaways

A practical recap of the architecture, routing, application dependencies, failure planning and cloud connectivity decisions behind a resilient multi-site data center.

Active-active data center design aims to use multiple sites to serve workloads concurrently while maintaining service continuity when links, network devices, applications or entire sites fail. Achieving this requires more than duplicating infrastructure: routing, application state, data consistency, security and recovery procedures must work together.

01

Architecture and workload placement

Define which services operate at each site, how traffic is distributed, and what happens when one location becomes unavailable. Active-active is a service and workload design decision, not simply a topology with two data centers.

02

Multi-site routing and BGP

Use explicit routing policy, controlled prefix advertisement and deliberate path selection. Validate convergence, asymmetric paths, route withdrawal and the capacity of the surviving site under failure conditions.

03

Layer 2 extension and overlays

Choose between routed inter-site connectivity, selective Layer 2 extension and overlay-based approaches according to workload requirements. Avoid extending broadcast domains across sites without a clear operational and failure-containment justification.

04

Application and data consistency

Understand replication lag, transaction behaviour, quorum, write ownership and recovery objectives. Network reachability alone cannot guarantee consistent data or prevent conflicting writes during a partition.

05

Failure detection and recovery

Monitor routing and infrastructure alongside application transactions, replication health and user-visible service availability. Test realistic failures and document safe failover, failback and recovery procedures.

06

Cloud connectivity and modern choices

Select VPN, private connectivity, network peering or hub-based connectivity based on reachability, routing, security and resilience requirements. Confirm provider-specific route propagation and service limitations before implementation.

Six engineering principles to carry into your design

  • Design around service availability, not device availability alone.
  • Ensure the surviving site has sufficient capacity for its intended failure role.
  • Keep routing policy, application behaviour and data consistency aligned.
  • Use Layer 2 extension only where the workload genuinely requires it.
  • Measure end-to-end service health, not just link and routing status.
  • Prove failover and recovery through repeatable tests and documented outcomes.

Final perspective: resilience must be engineered and tested

The strongest active-active designs make failure behaviour explicit. They define how traffic moves, how stateful applications continue safely, how data is protected, and how operators recognise and resolve degraded service. They also account for partial failures, where a site remains reachable but can no longer safely perform its intended role.

No single technology guarantees resilience. BGP, overlays, replication, cloud connectivity and redundant infrastructure are building blocks whose effectiveness depends on the requirements, implementation and operational testing around them.

The central takeaway

Build for the failure you need to survive, define the service outcome you must preserve, and test that outcome before relying on the architecture in production.

Network Insight · Data Center Architecture · Routing · High Availability · Cloud Networking
```
☕
Support Network Insight
Help support independent engineering guides, interactive networking tools, and future learning labs.
Support Network Insight ```