Spine–Leaf Architecture: Designing Scalable Data Center Fabrics
Modern data centers must move increasing volumes of traffic between distributed applications, virtual machines, containers and storage systems. Traditional hierarchical networks can struggle to deliver the bandwidth, predictable latency and flexibility these workloads demand. Spine–leaf architecture addresses these challenges through a scalable, multi-path network fabric.
Also known as a folded-Clos architecture, a conventional two-tier spine–leaf fabric consists of leaf switches that connect servers and other endpoints, and spine switches that interconnect the leaf layer. Each leaf connects to every spine, creating multiple equal-cost paths between endpoints attached to different leaves. Routing protocols and Equal-Cost Multipath (ECMP) forwarding allow eligible traffic flows to use these alternative paths.
This design is particularly suited to east–west traffic: communication between application servers, distributed services, virtualised workloads and storage systems inside the data center. Rather than forcing traffic through multiple hierarchical aggregation layers, the fabric provides a consistent path structure that can improve capacity planning, simplify expansion and make network behaviour easier to reason about.
However, a spine–leaf diagram alone does not guarantee a non-blocking network. Switch forwarding capacity, link speeds, oversubscription, ECMP hashing, routing convergence and application traffic patterns all influence actual performance. Resilient designs must account for both normal operating conditions and the capacity remaining after a component fails.
Move from topology fundamentals to real engineering decisions: compare traditional and spine–leaf designs, calculate oversubscription, trace ECMP paths, investigate failures, examine routed underlays and VXLAN/EVPN overlays, and understand how telemetry helps engineers diagnose and optimise a data center fabric.
Explore the Spine–Leaf Architecture Guide
Move from the fundamentals of fabric design to capacity planning, resilience, overlays and real-world operations. Select any section to jump directly to its explanation.
Traditional Three-Tier vs. Spine–Leaf Architecture
Data center architecture determines how traffic moves between servers, how bandwidth is distributed, and how the network responds as workloads grow. Understanding the difference between traditional hierarchical designs and spine–leaf fabrics is the starting point for making informed capacity, resilience and routing decisions.
The traditional three-tier model
Traditional data centers commonly use three logical network layers: access, aggregation and core. Access switches connect servers and endpoints. Aggregation switches collect access-layer traffic and may provide Layer 3 routing, policy enforcement and connectivity between network segments. Core switches provide high-speed transport between aggregation blocks and other network domains.
This hierarchy can work well for traffic patterns that move predominantly between access networks and external destinations. However, the aggregation and core layers can become capacity constraints when large numbers of distributed applications exchange traffic internally. Scaling may require additional uplinks, larger chassis or redesigns of existing aggregation blocks.
Three-tier hierarchy
Multiple hierarchical layers can introduce additional hops and concentrate traffic on aggregation uplinks.
Two-tier spine–leaf
Each leaf connects to every spine, providing equal-cost paths between different leaves in a conventional full fabric.
Why the design approach changed
Virtualisation, cloud platforms, distributed databases and containerised applications have increased the importance of east–west traffic—communication between workloads inside the data center. An application request may trigger multiple exchanges between services, storage systems and compute nodes before a response is returned.
Spine–leaf architecture addresses this pattern through a regular, repeatable topology. Endpoints connect to leaf switches, and each leaf connects to every spine. Traffic between endpoints attached to different leaves normally travels from the source leaf to a spine and then to the destination leaf. In a conventional two-tier fabric, this provides a predictable inter-leaf hop count.
Multiple routed paths allow Equal-Cost Multipath (ECMP) to forward eligible flows across available links. This can improve aggregate bandwidth utilisation and provide alternative paths when links or spine devices fail. The actual result depends on link capacity, routing convergence, traffic distribution and the fabric's remaining resources.
Key architectural differences
| Design consideration | Traditional three-tier | Spine–leaf fabric |
|---|---|---|
| Topology | Hierarchical access, aggregation and core | Leaf and spine switching tiers |
| Inter-leaf path | May traverse aggregation and core layers | Normally one spine between different leaves |
| Multipath forwarding | Depends on the design and supported protocols | Commonly uses ECMP over routed links |
| Scaling | May require uplink or hierarchical redesign | Add leaves or spines within design limits |
| Capacity planning | Focuses on hierarchical uplinks and aggregation | Focuses on leaf uplinks, spine capacity and oversubscription |
| Failure behaviour | Depends on redundancy and available alternate paths | Remaining ECMP paths can maintain connectivity if reachable |
What engineers must still consider
Spine–leaf is not automatically non-blocking, congestion-free or fault-tolerant in every deployment. The architect must calculate uplink capacity, understand oversubscription, verify the forwarding capabilities of the switches and test the consequences of losing a spine or a leaf uplink. Redundant server connections also require an appropriate design, such as supported LACP or multichassis link aggregation where needed.
The key difference is not simply two tiers instead of three. Spine–leaf creates a regular multipath fabric that makes inter-leaf connectivity and scale-out capacity more predictable. Good engineering still depends on the routing design, available bandwidth, failure strategy and workload requirements.
Compare the Network Architectures
Switch architectures, select network devices to inspect their roles, and animate an illustrative server-to-server path. This simplified diagram explains the design concepts rather than representing a vendor-specific production configuration.
Leaf switches
Leaf switches connect servers and provide uplinks into the spine layer.
Traffic animation is illustrative. Actual forwarding paths depend on routing, ECMP hashing, link state, topology and the configured network.
Spine–Leaf Fabric Architecture: Routing, ECMP and Traffic Flow
The defining feature of a spine–leaf network is its regular connectivity pattern. Every leaf switch connects to every spine switch, while servers and other endpoints attach to the leaf layer. This creates a predictable physical fabric in which routing and multipath forwarding determine how traffic reaches its destination.
Understanding the role of each layer
Leaf switches provide endpoint connectivity. They commonly connect to physical servers, virtualisation hosts, storage systems, firewalls and external network services. Depending on the design, leaf switches may also host Layer 3 gateway functions and VXLAN tunnel endpoints (VTEPs).
Spine switches provide the transit fabric between leaves. In a routed Layer 3 design, each leaf-to-spine connection is normally a point-to-point routed link. Spine switches forward traffic between the relevant leaf-facing interfaces without requiring the spine tier to act as a shared Layer 2 segment.
Leaf layer
Connects endpoints, advertises reachable routes and forwards traffic towards the destination leaf or an upstream service.
Spine layer
Interconnects the leaf switches and provides multiple transit paths across the fabric.
Underlay control plane
Protocols such as eBGP or OSPF establish Layer 3 reachability. The protocol and route design depend on the chosen fabric.
How a packet crosses the fabric
Consider Server A connected to Leaf 1 and Server B connected to Leaf 2. When Server A sends traffic to Server B, Leaf 1 performs its forwarding lookup and selects an appropriate next hop. In a conventional routed fabric, the packet travels across one of the available leaf-to-spine links. The selected spine then forwards it across its connection to Leaf 2, which delivers the packet to Server B.
Endpoints attached to the same leaf may communicate directly through that switch when the destination is locally reachable. Traffic between different leaves normally traverses a spine. This regular structure provides a consistent inter-leaf hop count, although the actual latency still depends on link speeds, congestion, forwarding performance and workload behaviour.
Equal-Cost Multipath (ECMP)
ECMP allows a router to install multiple next hops for a destination when the routing protocol identifies paths with equal routing cost. In a spine–leaf fabric, a leaf can commonly reach a destination prefix through multiple spine switches. Traffic can therefore use several available paths instead of depending on a single uplink.
Many implementations use a hash based on packet or flow fields to select a next hop. This typically keeps packets from a flow on the same path, helping preserve packet ordering. Consequently, ECMP does not guarantee that every link receives exactly the same amount of traffic: hash distribution, flow sizes and traffic patterns can create uneven utilisation.
| Engineering factor | What it means |
|---|---|
| Equal routing cost | Multiple next hops are eligible for forwarding to a destination according to the routing implementation. |
| Flow hashing | Traffic is mapped to paths using selected header fields; one large flow may dominate an individual link. |
| Available path count | The number of usable paths depends on topology, routing policy, installed routes and hardware limits. |
| Failure convergence | Following a failure, routes and forwarding state must update before traffic can reliably use alternate paths. |
Underlay and overlay: two different responsibilities
The underlay provides IP reachability across the physical fabric. In a VXLAN/EVPN design, leaf switches commonly advertise VTEP loopback reachability through the underlay. The overlay carries tenant or workload traffic using VXLAN encapsulation between VTEPs, while EVPN can distribute endpoint and reachability information.
Keeping these functions distinct helps engineers troubleshoot connectivity. A workload communication problem may arise from underlay routing, overlay control-plane information, endpoint learning, policy enforcement or the physical links themselves. Verifying each layer separately narrows the fault domain.
Spine–leaf provides a repeatable physical topology; routing establishes reachability, and ECMP enables multipath forwarding. Predictable performance still requires careful capacity planning, balanced traffic distribution and validation of failure behaviour.
Trace a Route Through the Spine–Leaf Fabric
Follow traffic from Server A to Server B. Compare two illustrative equal-cost paths, inspect the role of each device, and step through forwarding. The example assumes a routed underlay with ECMP-capable paths between the leaf switches.
Leaf 1 receives traffic
Leaf 1 receives the packet from Server A and performs a forwarding lookup. The route and next-hop information determine how it proceeds.
Important: ECMP makes eligible equal-cost next hops available; it does not guarantee that every flow or link carries exactly the same amount of traffic. The forwarding choice depends on the routing table, hashing algorithm, flow characteristics and current topology.
Spine–Leaf Capacity Planning: Bandwidth, Oversubscription and Growth
A spine–leaf fabric can provide predictable paths and substantial aggregate bandwidth, but its real capacity depends on link speeds, the number of uplinks, traffic distribution and the amount of bandwidth required during failures. Capacity planning must account for both normal operation and degraded conditions.
What oversubscription actually means
Oversubscription compares the bandwidth offered by a group of downstream interfaces with the bandwidth available through its upstream links. It is a useful design indicator, not a prediction that every link will be congested. Actual utilisation depends on workload demand, traffic locality, flow sizes and how traffic is distributed across available paths.
Nominally non-oversubscribed
The defined downstream capacity equals the available upstream capacity. This does not guarantee zero contention in every traffic pattern or at every point in the network.
Oversubscribed fabric
The defined downstream capacity is three times the upstream capacity. The design relies on demand not requiring every downstream interface to transmit at full rate simultaneously.
Capacity matched to demand
The target ratio should reflect measured traffic, service objectives, growth expectations and the cost of additional links.
Worked example: calculating leaf uplink capacity
Consider a leaf switch with 48 server-facing ports operating at 25 Gbps and four uplinks to the spine layer, each operating at 100 Gbps. This simplified example assumes all stated ports can transmit at their nominal line rate and compares aggregate downstream port capacity with aggregate uplink capacity.
The resulting 3:1 ratio means the aggregate server-facing port capacity is three times the aggregate uplink capacity. It does not mean the leaf will always experience congestion. If most traffic remains local to the leaf, less bandwidth may need to cross the fabric. If many servers simultaneously send large flows to endpoints on other leaves, uplinks can become a bottleneck.
This example also excludes protocol overhead, port breakout constraints, oversubscription elsewhere in the network and implementation-specific forwarding limits. Production designs should use the actual hardware configuration and expected traffic patterns rather than relying on this ratio alone.
Plan for failures, not just normal operation
Aggregate capacity changes when a link or spine fails. If a leaf has four 100 Gbps uplinks, losing one leaves three uplinks, or 300 Gbps of nominal uplink bandwidth, assuming the remaining links and paths are operational and usable. The example's downstream-to-uplink ratio therefore rises from 3:1 to 4:1.
The actual impact depends on the failure location, routing convergence, ECMP next-hop changes and the distribution of traffic across the surviving links. A resilient design must consider whether the remaining fabric can support critical workloads while operating in this reduced-capacity state.
| Design scenario | Capacity question | What to validate |
|---|---|---|
| Normal operation | Can the fabric support expected peak demand? | Utilisation, flow distribution, latency and congestion. |
| Single uplink failure | Can remaining leaf uplinks carry priority traffic? | Surviving bandwidth, routing convergence and ECMP rebalancing. |
| Spine failure | Can the remaining spines handle redistributed traffic? | Remaining path capacity, failure-domain impact and link utilisation. |
| Growth phase | Will new endpoints exceed the planned fabric capacity? | Port availability, uplink capacity, spine capacity and expansion limits. |
Capacity planning is an ongoing engineering task
Design decisions should be revisited using interface counters, telemetry, flow records and application performance measurements. Monitor sustained utilisation as well as short-lived bursts, packet drops, queue behaviour and latency. A fabric can show moderate average utilisation while still experiencing congestion during brief high-demand periods.
Growth planning should also include the physical limits of the chosen platform: available ports, supported speeds, optics, cabling, power, cooling and the number of endpoints that can be added without redesigning the fabric.
Calculate oversubscription explicitly, validate the ratio against real workload behaviour, and repeat the capacity calculation after relevant failures. A well-designed spine–leaf fabric is not simply one with many links; it is one whose available bandwidth meets service requirements under both normal and degraded conditions.
Calculate Leaf Oversubscription
Change the number and speed of server-facing ports and the uplinks connecting a leaf to the spine layer. See the nominal bandwidth ratio, then simulate the loss of one uplink to understand how redundancy affects available capacity.
1. Configure the leaf switch
Start with a typical example, then adjust the values to explore different port-density and uplink designs.
2. Configure the fabric uplinks
Uplink capacity is the sum of the configured uplink line rates. These controls assume all configured links are available before failure.
48 × 25 Gb/s server ports provide 1,200 Gb/s of nominal server-facing line rate. Four × 100 Gb/s uplinks provide 400 Gb/s of nominal uplink line rate, giving a 3:1 ratio.
3. Review the capacity results
Values are nominal line-rate calculations per direction, not a prediction of real application throughput.
The aggregate server-facing line rate is three times the nominal uplink line rate. Actual contention depends on the traffic pattern and active demand.
This calculation removes one uplink's capacity. It assumes the remaining links can continue forwarding; real convergence and link distribution depend on the platform and routing design.
Engineering note: Ethernet is full-duplex, so line rate is normally stated per direction; do not double these values when calculating a one-direction oversubscription ratio. The ratio alone does not establish a safe or unsafe design. Validate expected traffic, burst behaviour, ECMP distribution, failure scenarios, buffer behaviour and application requirements.
Failure Domains, Resilience and High Availability
A resilient spine–leaf fabric is designed to keep essential communication working when links or devices fail. Its regular topology provides multiple potential paths between leaves, but redundancy alone does not guarantee uninterrupted service. Routing convergence, remaining capacity, device dependencies and the location of the failure all influence the outcome.
Understand the main failure domains
A failure domain is the part of a network whose operation may be affected by a particular fault. A single link failure may have a limited impact, while a failed leaf can disconnect the servers attached to it. A spine failure may affect traffic across multiple leaves, even when the fabric has alternative spines available.
One path is lost
If other usable paths remain, routing can select alternate next hops. Available bandwidth falls, and traffic may need to redistribute across the surviving links.
Fabric capacity is reduced
Leaves may continue communicating through the remaining spines. The remaining fabric must have sufficient capacity to carry the traffic that is redirected onto it.
Attached endpoints are at risk
Servers connected only to the failed leaf can lose network connectivity. Dual-homing or redundant attachment may help, but only when the complete design supports it.
How ECMP helps the fabric recover
Equal-Cost Multipath (ECMP) allows a router to install multiple eligible next hops with the same routing cost. Traffic can then be distributed across those paths, commonly using a hash of packet or flow fields. If a next hop becomes unavailable and the routing system detects the change, the forwarding table can be updated to use the remaining eligible paths.
ECMP does not make every flow move instantly or guarantee perfectly even link utilisation. The result depends on the routing protocol, failure detection, convergence behaviour, hash inputs, flow sizes and the forwarding hardware. A small number of large flows can produce uneven utilisation even when several paths are available.
Redundancy must include enough surviving capacity
Consider a leaf with four 100 Gbps uplinks to separate spines. Before a failure, the nominal aggregate uplink capacity is 400 Gbps. If one uplink becomes unavailable, the remaining three links provide 300 Gbps of nominal aggregate capacity, assuming each remaining link is usable at its stated rate.
That reduction matters even if routing converges successfully. If traffic demand exceeds the capacity available after the failure, congestion or packet loss may occur. Capacity planning should therefore evaluate expected demand with a link or spine unavailable, not just the normal operating state.
| Failure | Potential consequence | Design and test focus |
|---|---|---|
| Leaf-to-spine link | Fewer uplinks and reduced capacity from the affected leaf. | Confirm alternate routes, detection time and surviving uplink utilisation. |
| Entire spine | Multiple leaves lose one set of paths through the fabric. | Check remaining spine capacity and traffic redistribution across the fabric. |
| Leaf switch | Endpoints attached only to that leaf may become unreachable. | Validate server attachment redundancy, gateway placement and application recovery. |
| Power or shared dependency | Several devices or links may fail together. | Review power feeds, cabling paths, rack placement and common failure dependencies. |
Design for independent failure domains
Two links do not necessarily provide independent protection if they share a physical path, power source or intermediate device. Likewise, connecting a server to two switches does not automatically create active-active resilience. The server NIC configuration, switching design, gateway behaviour and routing or multi-chassis technology must work together.
Depending on the architecture, server redundancy may use routed connectivity, Layer 2 link aggregation, a multi-chassis link aggregation mechanism or another supported host-attachment design. Choose the method according to platform capabilities and the required failure behaviour rather than assuming that one method fits every fabric.
Test recovery instead of assuming it
A practical resilience test should deliberately remove a link or device in a controlled environment, observe routing and forwarding changes, and measure the effect on representative application traffic. Check packet loss, recovery time, latency, route changes and the utilisation of surviving links. Repeat the test for the failures that matter to the service.
A resilient spine–leaf fabric needs alternate paths, working failure detection, correct routing behaviour and sufficient surviving capacity. Validate the whole system—including server attachment and shared physical dependencies—under realistic failure conditions. Redundant topology is the starting point; tested recovery is the outcome that matters.
What Happens When a Fabric Component Fails?
Select a failure scenario to see how a simplified routed spine–leaf fabric is affected. You can also select an individual spine or leaf to inspect its role. Green links represent available connections; red links indicate failed or affected connections.
Both spine paths are available in this simplified topology. Actual traffic distribution depends on routing state, ECMP hashing and link capacity.
Spine–leaf resilience
Select a device in the diagram to inspect its role. Use the failure controls to explore how losing a component changes the available paths.
This is a conceptual failure model, not a live network simulation. It assumes a two-spine, two-leaf routed fabric with independent links and enough remaining capacity to carry traffic after a single spine or uplink failure. Real outcomes depend on topology, routing convergence, link speeds, platform limits, endpoint attachment and traffic demand. A failed leaf can disconnect endpoints attached only to that leaf.
Underlay, VXLAN/EVPN and Cisco ACI
Spine–leaf describes a physical and logical fabric topology, but it does not prescribe a single way to provide network services. A common design uses a routed Layer 3 underlay to connect the switches and an overlay to provide logical Layer 2 or Layer 3 connectivity across that fabric. Cisco ACI is another approach, combining a spine–leaf fabric with a policy-driven architecture and centralised controller-based management.
Separate transport from network services
The underlay and overlay solve different problems. The underlay provides IP reachability between the relevant fabric endpoints, while an overlay can create logical network segments that are independent of the physical location of a workload. Keeping these roles distinct makes the architecture easier to reason about, operate and troubleshoot.
Physical fabric
Leaf and spine switches, server-facing interfaces, fabric links, transceivers, cabling and physical capacity. This layer determines the available ports, speeds and physical paths.
Routed underlay
IP addressing, routing adjacencies and routes that provide reachability across the fabric. Common designs use an IGP or BGP, depending on the operational model and requirements.
Overlay and segmentation
VXLAN can encapsulate tenant traffic across the IP fabric. EVPN commonly distributes MAC and IP reachability information and supports control-plane functions for VXLAN overlays.
Policy and operations
Network policy, tenant segmentation, automation, telemetry and troubleshooting workflows define how the fabric is configured, monitored and managed.
What VXLAN and EVPN contribute
VXLAN provides an encapsulation mechanism that carries an inner Ethernet frame across an IP network. The outer IP packet travels through the underlay, allowing logical Layer 2 segments to span multiple switches without requiring the same physical Layer 2 segment to extend across the entire fabric.
EVPN is a control-plane mechanism commonly used with VXLAN to advertise endpoint reachability and support MAC learning and IP-related information. It can help the fabric make forwarding decisions without relying solely on data-plane flooding and learning. Exact behaviour depends on the EVPN design, device capabilities and configuration.
This is an illustrative forwarding sequence, not a claim that every packet must cross a spine. Traffic between endpoints on the same leaf may be forwarded locally, depending on the network and forwarding configuration. Traffic between different leaves typically traverses the routed fabric.
Why the underlay still matters
An overlay does not remove the need for a reliable underlay. Tunnel endpoints must have the IP reachability required to communicate, and the physical fabric must provide sufficient bandwidth and predictable forwarding. If underlay routes, interfaces or MTU settings are incorrect, overlay services can fail even when their logical configuration appears valid.
Routing and next hops
Verify interface addressing, routing adjacencies, route installation and reachability between relevant tunnel endpoints.
MTU and packet handling
Account for encapsulation overhead and verify that the path supports the required packet sizes without unintended drops.
ECMP and capacity
Check usable next hops, traffic distribution, link utilisation and whether surviving paths can support the workload.
Where Cisco ACI fits
Cisco Application Centric Infrastructure (ACI) uses a spine–leaf fabric with a policy-driven operational model. In a typical ACI deployment, the Application Policy Infrastructure Controller (APIC) provides centralised policy and fabric management. The fabric switches enforce the configured policy and perform the forwarding functions supported by the platform.
ACI is not simply another name for VXLAN/EVPN. It is a broader architecture with its own policy model, management components and control mechanisms. By contrast, a conventional spine–leaf network can be assembled using routed switching and separately selected overlay and automation technologies.
| Design area | Routed spine–leaf with VXLAN/EVPN | Cisco ACI |
|---|---|---|
| Architecture | A routed fabric with an overlay and control-plane choices selected for the deployment. | An integrated fabric architecture with a policy-driven model. |
| Configuration model | Commonly uses switch configuration, automation and selected management or controller tools. | Uses ACI constructs and APIC-based policy and fabric management. |
| Segmentation | Implemented through the selected overlay, routing and policy design. | Implemented through ACI policy constructs and supported fabric features. |
| Operational considerations | Requires a consistent design and operational model across the selected protocols and tools. | Requires understanding of ACI policy, fabric operation, controller health and platform-specific behaviour. |
Neither approach is universally better. Selection depends on operational skills, policy requirements, interoperability, automation, platform constraints, lifecycle costs and the organisation's preferred way of managing change.
Keep the roles clear: the physical fabric provides connectivity, the routed underlay provides IP reachability, and an overlay such as VXLAN/EVPN can provide logical network services. Cisco ACI packages fabric operation and policy into a broader architecture. Whichever model you choose, validate routing, encapsulation, MTU, segmentation and failure behaviour as one end-to-end system.
Trace a Packet Through Underlay and Overlay
A modern data-centre fabric can separate IP transport from tenant connectivity. Explore the routed underlay, the VXLAN data plane and the EVPN control plane, then step through an illustrative packet journey between endpoints connected to different leaf switches.
Routed underlay
Provides IP reachability between VTEP addresses across the leaf–spine fabric. Routing and ECMP select available transport paths.
VXLAN overlay
Encapsulates an inner Ethernet frame in an outer IP/UDP/VXLAN packet, carrying a VNI to identify the overlay segment.
EVPN control plane
Uses BGP EVPN routes to distribute MAC and, where applicable, IP reachability information so VTEPs can learn remote endpoints.
Encapsulate at the source VTEP
Leaf 1 receives Endpoint A's Ethernet frame, identifies the tenant segment and encapsulates the frame in VXLAN for the destination VTEP on Leaf 2.
Conceptual example only. The IP addresses and VNI are illustrative. Exact EVPN route types, gateway placement, ARP/ND handling, multicast or ingress-replication behaviour and policy depend on the implementation. Cisco ACI is a distinct policy-driven fabric architecture; it should not be treated as simply another name for a generic VXLAN/EVPN deployment.
Telemetry, Analytics and Troubleshooting a Spine–Leaf Fabric
A spine–leaf fabric can have many equal-cost paths, high aggregate bandwidth and redundant links, yet applications can still suffer from packet loss, latency or intermittent connectivity. Effective operations combine physical interface health, routing state, forwarding behaviour and application-level evidence to locate the actual fault rather than treating every symptom as a switch problem.
Build visibility across the fabric
No single monitoring source explains every network problem. Interface counters can reveal physical errors and congestion; routing information shows which paths are eligible; flow telemetry helps explain where traffic is going; and application measurements show whether the network is meeting service requirements. Correlating these sources provides a more reliable picture than examining isolated counters or alerts.
Interfaces and optics
Monitor link state, CRC errors, discards, interface resets, transceiver alarms and changes in error counters over time.
Routing and neighbours
Check routing adjacencies, installed routes, next-hop changes, ECMP membership and evidence of repeated convergence.
Flows and utilisation
Identify heavy flows, asymmetric traffic patterns, sustained utilisation, bursts, queue drops and uneven path loading.
Latency and reachability
Correlate network events with application latency, packet loss, connection failures and the timing of user-visible incidents.
Know what each monitoring source can tell you
| Source | Useful evidence | Important limitation |
|---|---|---|
| SNMP or interface polling | Interface status, byte and packet counters, errors and discards. | Polling intervals may miss short bursts; counter changes need context. |
| Streaming telemetry | Frequent structured measurements of supported interface, queue, routing or device state. | Available metrics and sampling behaviour vary by platform and configuration. |
| Flow records | Traffic conversations, source and destination patterns, protocol usage and traffic-volume estimates. | Sampling and export configuration affect what is visible; flow records do not necessarily explain every packet drop. |
| Routing and forwarding tables | Selected routes, next hops, ECMP paths and forwarding entries. | A valid route does not prove that the physical path is healthy or that application traffic is performing well. |
| Active probes and application monitoring | Reachability, round-trip time, loss and application response behaviour. | Results depend on probe placement, test traffic and the path taken by the measured flow. |
A systematic troubleshooting workflow
Start with the observed symptom and narrow the scope. Determine which endpoints, applications, leaves and paths are affected. Establish when the issue began, compare current state with a known-good baseline, and look for changes that match the incident timeline.
Define the impact
Identify affected source and destination endpoints, the application or service, whether the issue is constant or intermittent, and whether it affects one leaf or multiple leaves.
Check physical health
Inspect interface state, error counters, discards, optics, cabling and queue statistics along the likely path. Compare counter changes over a consistent time interval.
Verify routing and ECMP
Confirm reachability and installed routes. Check whether the expected next hops are present, whether adjacencies are stable and whether a recent failure changed path selection.
Examine traffic and congestion
Compare utilisation across relevant uplinks, inspect available queue and drop metrics, and look for large flows or uneven ECMP distribution. High average utilisation is not the only possible cause of congestion.
Correlate with application evidence
Compare network events with application latency, retransmissions, connection errors and probe results. Check endpoint, host and service health before concluding that the fabric is responsible.
Verify the fix
Repeat the same tests, confirm counters stop increasing abnormally, validate the intended paths and check application performance. Record the cause and the evidence that demonstrated recovery.
Example: diagnosing intermittent application latency
Imagine that an application communicates between servers connected to different leaf switches. Users report occasional delays, but the links remain up and routing adjacencies appear stable. The correct response is to gather evidence across several layers rather than assuming that the fabric has sufficient capacity because no interface is continuously saturated.
| Observation | Possible interpretation | Next check |
|---|---|---|
| Interface errors increase | A physical link, optic, cable or interface issue may be present. | Identify the affected interface and inspect physical diagnostics and error-counter changes. |
| Queue drops rise during bursts | Short periods of congestion may be affecting latency or packet delivery. | Correlate queue statistics with traffic bursts and the affected flows. |
| One uplink is much busier than others | ECMP hashing or a small number of large flows may create uneven utilisation. | Inspect available flow data, hash behaviour and per-link counters. |
| Routes change near the incident time | A link event or routing instability may have altered the forwarding path. | Review logs, adjacency events, route changes and convergence timing. |
| Fabric indicators appear normal | The cause may lie at the host, virtual switch, application or another dependency. | Compare endpoint metrics, service logs, packet captures and independent probes. |
Build useful baselines and alerts
Alerting is more effective when it combines thresholds with trends and context. A brief utilisation spike may be expected, while a sustained rise in discards or recurring routing changes may deserve immediate investigation. Establish baselines for normal and peak periods, and tune alert thresholds to the platform and service requirements.
Useful operational dashboards bring together topology, interface health, link utilisation, routing state, device events and service performance. Time synchronisation across devices and monitoring systems is essential when correlating an incident. Retain enough historical data to compare the period before, during and after the fault.
Troubleshoot from evidence, not assumptions. Correlate physical health, routing and ECMP state, traffic distribution, congestion indicators and application symptoms. The aim is to identify the fault domain, demonstrate the cause with observable data, and verify that the corrective action restores the required service.
Diagnose a Spine–Leaf Fabric Incident
Choose a symptom, inspect the simulated telemetry and work through the diagnostic sequence. The objective is to identify evidence that supports a hypothesis before changing the network.
Application response times rise during busy periods. Determine whether congestion, physical errors or another fault is supported by evidence.
Establish the scope before changing anything
Identify which applications, source and destination endpoints, paths and time windows are affected. Compare the symptoms with a known-good baseline.
Training simulation: all metrics and fault indicators are illustrative, not live device telemetry. High utilisation alone does not prove congestion; correlate it with drops, queue behaviour, latency and the affected traffic. A route being present does not prove the complete data path is healthy.
Spine–Leaf Design Decisions and Implementation Checklist
A successful spine–leaf deployment starts with workload requirements, not a switch count or a preferred topology diagram. The design should establish the required connectivity, performance, resilience and operational model before hardware, routing protocols and overlays are selected. Each decision should be validated against the services the fabric must support.
Start with requirements, not assumptions
Document how many endpoints must connect, where workloads communicate, which applications are sensitive to latency, and how much traffic is expected to cross between leaves. Include planned growth, maintenance requirements, failure scenarios and operational constraints. These inputs determine the appropriate topology, link speeds, oversubscription target and expansion strategy.
Endpoint and port needs
Estimate server counts, port speeds, dual-homing requirements, network segments and the number of leaf switches needed.
Traffic and latency
Understand east–west traffic, peak demand, workload locality, latency targets and expected flow distribution.
Failure requirements
Define which failures the fabric must tolerate and what performance is required while operating in a degraded state.
Management and skills
Plan configuration, automation, monitoring, troubleshooting, maintenance windows and team expertise.
Work through the design in a controlled sequence
Choose the physical topology
Determine the number of leaves and spines, required endpoint density, cabling paths and supported link speeds. Check platform port limits and expansion constraints before finalising the topology.
Define the routed underlay
Select the addressing plan, routing protocol, link addressing, route policy and failure-detection approach. Keep the design consistent and document how routes should behave during faults.
Set capacity and oversubscription targets
Calculate aggregate server-facing bandwidth and uplink capacity. Test expected peak traffic, account for ECMP behaviour, and verify remaining capacity after relevant link or spine failures.
Select segmentation and overlay services
Decide whether routed connectivity alone meets requirements or whether VXLAN/EVPN, another overlay or a policy-driven fabric architecture is needed. Document Layer 2 and Layer 3 service boundaries and security controls.
Plan endpoint and gateway redundancy
Define server attachment, gateway placement, link aggregation or other supported redundancy mechanisms. Validate the complete design rather than assuming two connections guarantee continuity.
Build observability into the deployment
Establish telemetry, interface and queue monitoring, routing event collection, flow visibility, time synchronisation and application-level performance baselines.
Validate before production rollout
Test connectivity, expected forwarding paths, policy enforcement, failure recovery, degraded capacity and operational procedures. Record the results and resolve gaps before expanding the deployment.
Implementation readiness checklist
Use the following checklist during design review and pre-production validation. Each item should have supporting evidence, such as a calculation, configuration review, test result or documented decision.
Common design mistakes to avoid
| Assumption | Why it can be misleading | Better approach |
|---|---|---|
| More spines always solve capacity problems | Endpoint links, leaf uplinks, hardware limits or traffic distribution may remain the bottleneck. | Identify the actual constraint and recalculate end-to-end capacity. |
| ECMP guarantees equal utilisation | Hashing and flow-size differences can create uneven traffic distribution. | Inspect per-link counters and available flow telemetry under realistic loads. |
| Redundancy guarantees uninterrupted service | Convergence, shared dependencies, endpoint design and reduced capacity can still interrupt or degrade applications. | Define failure requirements and measure recovery through controlled tests. |
| An overlay removes underlay complexity | Tunnel endpoints still depend on routing, MTU, physical connectivity and correct forwarding state. | Troubleshoot underlay and overlay as separate but connected layers. |
| Average utilisation tells the whole story | Short bursts, queue congestion, packet drops or a few large flows can affect service even when averages look acceptable. | Combine time-series metrics, queue data, flow evidence and application measurements. |
Plan for the full lifecycle
The fabric should be designed for more than its initial deployment. Establish how new leaves will be added, how capacity will be reassessed, how software and hardware will be maintained, and how configuration changes will be tested. Define measurable service targets and periodically review whether the fabric still meets them as workload patterns and business requirements change.
A scalable architecture is one that can grow predictably without introducing unmanaged complexity. That requires consistent configuration, clear ownership, repeatable validation and operational evidence—not just a topology that looks symmetrical.
Design from workload requirements, calculate capacity explicitly, separate underlay and overlay responsibilities, and test failure behaviour before production. A robust spine–leaf fabric balances performance, resilience, operational simplicity and the cost of future growth.
Build a Spine–Leaf Design Readiness Plan
Configure a sample fabric, review its nominal bandwidth and consider what happens when an uplink fails. Then complete the readiness checklist to assess whether the design has been specified, tested and documented.
The drawing shows up to five representative spines. The configured spine count is used in the design inputs, not as proof of available switch capacity.
3. Implementation-readiness checklist
0 of 8 completeDesign review not yet complete
Use the capacity calculation as an initial check, complete the readiness items and validate the design against workload and failure requirements.
Capacity model assumptions: nominal port line rates are compared in one direction and are not doubled for full duplex. The model does not account for protocol overhead, internal switch fabric limits, traffic locality or real flow distributions. After one uplink fails, one uplink's capacity is removed. The topology is illustrative and does not prove that the selected switches have sufficient ports, bandwidth or forwarding capacity. Validate the physical design and required failure scenarios before deployment.
- BGP Solutions and Insights - October 8, 2026
- DMVPN - May 20, 2023
- Computer Networking: Building a Strong Foundation for Success - April 7, 2023