fabricpath design

Data Center Fabric

 

fabricpath design

 

Data Center Fabric

In today’s digital age, where vast amounts of data are generated and processed, data centers play a vital role in ensuring seamless and efficient operations. At the heart of these data centers lies the concept of data center fabric – a sophisticated infrastructure that forms the backbone of modern computing. In this blog post, we will delve into the intricacies of data center fabric, exploring its importance, components, and benefits.

Data center fabric refers to the underlying architecture and interconnectivity of networking resources within a data center. It is designed to efficiently handle data traffic between various components, such as servers, storage devices, and switches while ensuring high performance, scalability, and reliability. Think of it as the circulatory system of a data center, facilitating the flow of data and enabling seamless communication between different entities.

 

  • Example: Data Center Fabric – FabricPath

Network devices are deployed in highly interconnected layers, represented as a fabric. Unlike traditional multitier architectures, a data center fabric effectively flattens the network architecture, reducing the distance between endpoints within the data center. An example of a data center fabric is FabricPath.

Cisco has validated FabricPath as an Intra-DC Layer 2 multipath technology. Design cases are also available where FabricPath is deployed for DCI ( Data Center Interconnect ). Regarding a FabricPath DCI option, design carefully over short distances with reliable interconnects, for example, Dark Fiber or Protected Dense Wavelength Division Multiplexing (DWDM ).

FabricPath designs are suitable for a range of topologies, and unlike hierarchical virtual Port Channel ( vPC ) designs, FabricPath does not need to follow any topology. It can accommodate any design type; full mesh, partial mesh, hub, and spoke topologies.

 

  • Example: Data Center Fabric – Cisco ACI 

ACI Cisco is a software-defined networking (SDN) architecture that brings automation and policy-driven application profiles to data centers. By decoupling network hardware and software, ACI provides a flexible and scalable infrastructure to meet dynamic business requirements. It enables businesses to move from traditional, manual network configurations to a more intuitive and automated approach.

One of the defining features of Cisco ACI is its application-centric approach. It allows IT teams to define policies based on application requirements rather than individual network components. This approach simplifies network management, reduces complexity, and ensures that network resources are aligned with the needs of the applications they support.

 

Before you proceed, you may find the following posts helpful:

  1. What Is FabricPath
  2. Data Center Topologies
  3. ACI Networks
  4. Active Active Data Center Design
  5. Redundant Links

 



Data Center Fabric

Key Data Center Fabric Discussion Points:


  • Introduction to the data center fabric and what is involved.

  • Highlighting the details of FabricPath.

  • Critical points on the possible alternatives to FabricPath.

  • Technical details on the load balancing.

 

Back to basic with a data center fabric.

Key Components of Data Center Fabric:

1. Network Switches: Network switches form the core of the data center fabric, providing connectivity between servers, storage devices, and other networking equipment. These switches are designed to handle massive data traffic, offering high bandwidth and low latency to ensure optimal performance.

2. Cabling Infrastructure: A well-designed cabling infrastructure is crucial for data center fabric. High-speed fiber optic cables are commonly used to connect various components within the data center, ensuring rapid data transmission and minimizing signal loss.

3. Network Virtualization: Network virtualization technologies, such as software-defined networking (SDN), play a significant role in the data center fabric. By decoupling the network control plane from the physical infrastructure, SDN enables centralized management, improved agility, and flexibility in allocating resources within the data center fabric.

 

Flattening the network architecture

In this current data center network design, network devices are deployed in two interconnected layers, representing a fabric. Sometimes, massive data centers are interconnected with three layers. Unlike conventional multitier architectures, a data center fabric flattens the network architecture, reducing the distance between endpoints within the data center. This design results in high efficiency and low latency. Very well suited for east-to-west traffic flows.

Data center fabrics provide a solid layer of connectivity in the physical network, and move the complexity of delivering use cases for network virtualization, segmentation, stretched Ethernet segments, workload mobility, and various other services to an overlay that rides on top of the fabric.

When paired with an overlay, the fabric itself is called the underlay. The overlay could be deployed with, for example, VXLAN. To gain network visibility into user traffic, you would examine the overlay, and the underlay is used to route traffic between the overlay endpoints.

VXLAN, short for Virtual Extensible LAN, is a network virtualization technology that enables the creation of virtual networks over an existing physical network infrastructure. It provides a scalable and flexible approach to address the challenges posed by traditional VLANs, such as limited scalability, spanning domain constraints, and the need for manual configuration.

 

  • A key point: Lab guide on overlay networking with VXLAN

The following example shows VLXAN tunnel endpoints on Leaf A and Leaf B. The bridge domain is mapped to a VNI on G3 on both leaf switches. This enables a Layer 2 overlay for the two hosts to communicate. This VXLAN overlay goes across Spine A and Spine B.

Take note that the Spine layer, which acts as the core network, which would be a WAN network or any other type of Routed Layer 3 network, does not have any VXLAN configuration. We have flatted the network while providing Layer 2 connectivity over a routed core.

VXLAN overlay
Diagram: VXLAN Overlay

 

Fabricpath Design: Problem Statement

Key Features of Cisco Fabric Path:

Transparent Interconnection: Cisco Fabric Path allows for creating a multi-path forwarding infrastructure that provides transparent Layer 2 connectivity between devices within a network. This enables the efficient utilization of available bandwidth and simplifies network design.

Scalability: With Cisco Fabric Path, organizations can quickly scale their network infrastructure to accommodate growing data loads. It supports up to 16 million virtual network segments, enabling seamless expansion of network resources without compromising performance.

Fault Tolerance: Cisco Fabric Path incorporates advanced fault-tolerant mechanisms like loop-free topology and equal-cost multipath routing. These features ensure high availability and resiliency, minimizing the impact of network failures and disruptions.

2.4 Traffic Optimization: Cisco Fabric Path employs intelligent load-balancing techniques to distribute traffic across multiple paths, optimizing network utilization and reducing congestion. This results in improved application performance and enhanced user experience.

The problem with traditional classical Ethernet is the flooding behavior of unknown unicasts and broadcasts and the process of MAC learning. All switches must learn all MAC addresses, leading to inefficient resource use. In addition, Ethernet has no Time-to-Live ( TTL ) value, and if precautions are not in place, it could cause an infinite loop.

data center fabric
Diagram: The need for a data center fabric

 

Deploying Spanning Tree Protocol ( STP ) at Layer 2 blocks loops, but STP has many known limitations. One of its most significant flaws is that it offers a single topology for all traffic with one active forwarding path. Scaling the data center with classical Ethernet and spanning trees is inefficient as it blocks all but one path. With spanning trees’ default behavior, the benefits of adding extra spines do not influence bandwidth or scalability.

Possible alternatives

  • Multichassis EtherChannel 

To overcome these limitations, Cisco introduced Multichassis EtherChannel ( MEC ). MEC comes in two flavors; Virtual Switching System ( VSS ) with Catalyst 6500 series or Virtual Port Channel ( vPC ) with Nexus Series. Both offer active/active forwarding but present scalability challenges when scaling out Spine / Core layers. Additionally, complexity increases when deploying additional spines.

 

  • Multiprotocol Label Switching 

Another option would be to scale out with Multiprotocol Label Switching ( MPLS ). Replace Layer 2 switching with Layer 3 forwarding and MPLS with Layer 2 pseudowires. This type of complexity would lead to an operational nightmare. The prevalent option is to deploy Layer 2 multipath with THRILL or FabricPath. In intra-DC communication, Layer 2 and Layer 3 designs are possible in two forms; Traditional DC design and Switched DC design.

FabricPath VLANs use Conversational Learning, meaning a subset of MAC addresses is learned at the network’s edge. Conversation learning consists of a three-way handshake. Each interface learns the MAC addresses of interested hosts. Compared to classical Ethernet, each switch device learns all MAC addresses for that VLAN.

  1. Traditional DC design replaces hierarchical vPC and STP with FabricPath. Core, distribution, and access elements stay the same. The same layered hierarchical model exists but with FabricPath in the core.
  2. Switched DC design based on Clos Fabrics. Integrate additional Spines for Layer 2 and Layer 3 forwarding.

 

Traditional data center design

what is data center fabric
Diagram: what is data center fabric

 

Fabric Path in the core replaces vPC. It still uses port channels, but the hierarchical vPC technology previously used to provide active/active forwarding is not required. Instead, designs are based on modular units called PODs; within each POD, traditional DC technologies exist, for example, vPC. Active/active ( dual-active paths ) forwarding based on a two-node Spine, Hot Standby Router Protocol ( HSRP ), announces the virtual MAC of the emulated switch from each of the two cores. For this to work, implement vPC+ on the inter-spine peer links.

 

Switched data center design

Switched Fabric Data Center
Diagram: Switched Fabric Data Center

 

Each edge node has equidistant endpoints to each other, offering predictable network characteristics. From FabricPath’s outlook, the entire Spine Layer is one large Fabric-based POD. In the traditional model presented above, port and MAC address capacity are key factors influencing the ability to scale out. The key advantage of Clos-type architecture is that it expands the overall port and bandwidth capacity within each POD.

Implementing load balancing 4 wide spines challenges traditional First Hop Redundancy Protocol ( FHRP ) like HSRP, which works with 2 active pairs by default. Implementing load balancing 4 wide spines with VLANs allowed on certain links is possible but can cause link polarization

For optimized designs, utilize a redundancy protocol to work with a 4-node gateway. Deploy Gateway Load Balancing Protocol ( GLBP ) and Anycast FHRP. GLBP uses a weighting parameter that allows Address Resolution Protocol ( ARP ) requests to be answered by MAC addresses pointing to different routers. Anycast FHRP is the recommended solution for designs with 4 or more spine nodes.

 

FabricPath Key Points:

  • FabricPath removes the requirement for a spanning tree and offers a more flexible and scalable design to its vPC-based Layer 2 alternative. No requirement for a spanning tree, enabling Equal Cost Multipath ( ECMP ).

  • FabricPath no longer forwards using spanning tree. Offering designers bi-sectional bandwidth and up to 16-way ECMP. 16 x 10Gbps links equate to 2.56 terabits per second between switches.

  • Data Centers with FabricPath are easy to extend and scale.

  • Layer 2 troubleshooting tools for FabricPath including FabricPath PING and Traceroute can now test multiple equal paths.

  • Control plane based on Intermediate System-to-Intermediate System ( IS-IS ).

  • Loop prevention is now in the data plane based on the TTL field.