EVPN VXLAN Data Center Architecture

EVPN VXLAN Data Centre Architecture Explained

NETWORK INSIGHT · ENGINEERING GUIDE 16 OF 1,000

EVPN VXLAN Data Centre Architecture Explained

Modern data centres use routed leaf-spine fabrics and network overlays to deliver scalable workload connectivity, Layer 2 extension, Layer 3 segmentation, resilient forwarding and efficient east-west traffic movement. BGP EVPN provides the overlay control plane, while VXLAN carries encapsulated tenant traffic across an IP underlay.

This engineering guide explores the complete architecture of EVPN VXLAN networks, including VTEPs, VNIs, BGP EVPN route types, route reflectors, anycast gateways, integrated routing and bridging, multi-tenancy, convergence and real-world troubleshooting.

BGP EVPN VXLAN Leaf-Spine VTEPs L2VNI / L3VNI Anycast Gateway ECMP

Interactive EVPN VXLAN Fabric Architecture

ANIMATED CONTROL + DATA PLANE
SPINE LAYER · ROUTED IP UNDERLAY SPINE 1 IP Fabric / ECMP SPINE 2 IP Fabric / ECMP LEAF LAYER · VXLAN TUNNEL ENDPOINTS LEAF 1 VTEP 10.0.0.11 LEAF 2 VTEP 10.0.0.12 LEAF 3 VTEP 10.0.0.13 LEAF 4 VTEP 10.0.0.14 TENANT WORKLOADS · LOGICAL OVERLAY NETWORKS HOST A VNI 10010 HOST B VNI 10010 HOST C VNI 20020 HOST D VNI 20020 UNDERLAY IP REACHABILITY · OVERLAY EVPN ROUTES · VXLAN ENCAPSULATION
IP Underlay — Routed Leaf-Spine Connectivity

Every leaf connects to both spines. The routed underlay provides IP reachability between VTEP loopback addresses, allowing VXLAN packets to traverse the fabric using ECMP.

What Makes EVPN VXLAN Different?

ARCHITECTURE 01
Routed IP Underlay

The physical fabric uses Layer 3 routing and ECMP instead of extending spanning-tree domains throughout the data centre.

ARCHITECTURE 02
BGP EVPN Control Plane

MP-BGP distributes overlay reachability information, including MAC/IP advertisements and other EVPN routes.

ARCHITECTURE 03
VXLAN Data Plane

VTEPs encapsulate tenant Ethernet frames inside UDP/IP packets and forward them across the underlay.

How an EVPN VXLAN Packet Crosses the Fabric

Consider Host A on Leaf 1 communicating with Host B on Leaf 2. Both workloads belong to the same Layer 2 virtual network, identified by VNI 10010.

HOST A | | Original Ethernet frame v LEAF 1 / VTEP 10.0.0.11 | | VXLAN encapsulation | | Outer source IP: 10.0.0.11 | Outer dest IP: 10.0.0.12 | UDP dest port: 4789 | VXLAN VNI: 10010 | v ROUTED IP UNDERLAY | | ECMP-selected spine path v LEAF 2 / VTEP 10.0.0.12 | | VXLAN decapsulation v HOST B

BGP EVPN enables the originating VTEP to learn relevant remote MAC/IP reachability. The underlay then forwards the encapsulated packet toward the remote VTEP address.

What You Will Engineer in This Guide

The nine engineering sections progressively build the knowledge required to design, operate and troubleshoot production EVPN VXLAN fabrics.

FOUNDATIONS
Underlay and Overlay

Understand leaf-spine design, VTEPs, VNIs, IP routing and encapsulated forwarding.

CONTROL PLANE
BGP EVPN and Route Types

Investigate route advertisements, route reflectors, MAC/IP learning and EVPN policy.

PRODUCTION
Operations and Troubleshooting

Analyze anycast gateways, multi-tenancy, convergence, failures and forwarding anomalies.

Begin the EVPN VXLAN Engineering Journey Start with the architectural foundations of modern overlay-based data centre networks.
Explore Section 1 →
NETWORK INSIGHT · ENGINEERING GUIDE 16 OF 1,000

Explore the EVPN VXLAN Architecture

Navigate the nine engineering sections through an animated representation of the EVPN control plane, VXLAN overlay tunnels and routed leaf-spine fabric. Each section investigates a different part of the architecture, from foundational forwarding to production incident analysis.

SPINE 1 SPINE 2 LEAF 1 LEAF 2 LEAF 3 LEAF 4 EVPN BGP EVPN CONTROL PLANE · VXLAN DATA PLANE · IP UNDERLAY
IP Spine VTEP Leaf EVPN Control Plane Fabric Activity
Select a section below to navigate directly to its engineering content.
SECTION 01 · EVPN VXLAN ARCHITECTURE FUNDAMENTALS

Understanding the EVPN VXLAN Architecture

EVPN VXLAN combines a routed IP fabric with a virtualized network overlay. The physical underlay provides IP reachability between tunnel endpoints, while the overlay enables tenant Layer 2 and Layer 3 connectivity across the fabric.

The architecture separates transport forwarding from tenant reachability. BGP EVPN distributes overlay control-plane information, while VXLAN encapsulates tenant Ethernet frames for forwarding across the IP underlay.

1.1 — Why Modern Data Centres Use EVPN VXLAN

Traditional Layer 2 data centre networks often rely on spanning-tree-based topologies. Although suitable for many environments, large extended broadcast domains can introduce operational and scaling challenges.

Routed leaf-spine architectures use multiple equal-cost paths and a Layer 3 underlay. EVPN VXLAN adds the ability to provide logical Layer 2 segments across those routed boundaries without requiring the physical fabric to behave as one large Layer 2 network.

DESIGN DRIVER 01
Scalable Forwarding

The routed fabric supports multipath forwarding and avoids dependence on spanning tree for inter-switch fabric links.

DESIGN DRIVER 02
Logical Segmentation

VNIs and VRFs enable separation of tenant broadcast domains and routed network contexts.

DESIGN DRIVER 03
Control-Plane Learning

BGP EVPN distributes overlay reachability rather than relying exclusively on data-plane flooding.

1.2 — Interactive Underlay and Overlay Architecture

Select the architectural layer to inspect how physical IP routing, BGP EVPN signalling and VXLAN forwarding interact within the fabric.

EVPN VXLAN Layer Explorer

CONTROL PLANE + DATA PLANE
ROUTED SPINE FABRIC SPINE 1 IP Transit SPINE 2 IP Transit VTEP LEAF SWITCHES LEAF 1 10.0.0.11 LEAF 2 10.0.0.12 LEAF 3 10.0.0.13 LEAF 4 10.0.0.14 HOST A VNI 10010 HOST B VNI 10010 HOST C VNI 20020 HOST D VNI 20020
IP Underlay

Routed leaf-spine links provide reachability between VTEP loopbacks. ECMP allows the underlay to select among multiple equal-cost paths.

1.3 — The Role of the VXLAN Tunnel Endpoint

A VXLAN Tunnel Endpoint, or VTEP, is responsible for encapsulating and decapsulating VXLAN traffic. In many data centre deployments, leaf switches act as VTEPs.

When a frame enters a VXLAN segment, the ingress VTEP identifies the relevant VNI and remote destination VTEP, then encapsulates the original Ethernet frame inside an outer UDP/IP packet.

VXLAN FORWARDING PROCESS HOST A | v LEAF 1 / INGRESS VTEP | +-- Identify tenant segment +-- Determine VNI +-- Resolve remote VTEP +-- Add VXLAN + UDP/IP headers | v ROUTED IP UNDERLAY | +-- Forward using outer IP header | v LEAF 2 / EGRESS VTEP | +-- Remove VXLAN encapsulation +-- Forward original Ethernet frame | v HOST B

1.4 — VXLAN Network Identifiers

A VXLAN Network Identifier, or VNI, is a 24-bit identifier used to distinguish VXLAN segments. This provides a much larger identifier space than the 12-bit VLAN ID.

In an EVPN VXLAN fabric, Layer 2 VNIs represent logical Ethernet segments, while Layer 3 VNIs may be associated with tenant VRFs in symmetric IRB designs.

OVERLAY IDENTIFIER 01
Layer 2 VNI

Identifies a logical Ethernet broadcast domain across VTEPs. Used for same-subnet Layer 2 forwarding.

OVERLAY IDENTIFIER 02
Layer 3 VNI

Commonly identifies a tenant routed overlay context associated with a VRF in symmetric IRB architectures.

1.5 — Interactive VXLAN Packet Encapsulation Inspector

Select a packet header to understand how the original Ethernet frame is carried through the routed fabric.

VXLAN Encapsulation Stack

PACKET STRUCTURE EXPLORER
OUTER ETH Next-hop MACs
OUTER IP VTEP addresses
UDP Dst 4789
VXLAN VNI 10010
INNER FRAME Tenant Ethernet
Outer Ethernet Header

The outer Ethernet header is used for local hop-by-hop forwarding across each physical link. Its source and destination MAC addresses may change as the packet traverses routed hops.

1.6 — BGP EVPN as the Overlay Control Plane

VXLAN defines a method of encapsulating Ethernet frames over an IP network, but does not by itself define a complete distributed control plane for learning remote endpoint reachability.

BGP EVPN provides a standards-based mechanism for advertising relevant overlay information, including MAC/IP reachability and other EVPN route types.

BGP EVPN CONTROL-PLANE EXAMPLE LEAF 1 LEAF 2 Local Host A Local Host B MAC A MAC B VNI 10010 VNI 10010 BGP EVPN ROUTE EXCHANGE ----------------------- Leaf 1 learns: MAC B -> Remote VTEP 10.0.0.12 Leaf 2 learns: MAC A -> Remote VTEP 10.0.0.11 DATA PLANE: VXLAN encapsulation over IP CONTROL PLANE: MP-BGP EVPN reachability

1.7 — Underlay Versus Overlay Responsibilities

Component Underlay Overlay
Primary purpose Transport IP packets between VTEPs. Provide tenant network connectivity.
Routing eBGP, OSPF, IS-IS or another supported design. BGP EVPN reachability and VRF forwarding.
Identifiers IP prefixes and next hops. VNIs, MAC addresses, tenant IPs and route targets.
Forwarding IP forwarding and ECMP. VXLAN encapsulation and tenant forwarding.
Troubleshooting Links, adjacencies, routing and MTU. EVPN routes, VNI mapping and VTEP state.

1.8 — Architecture Design Considerations

A production EVPN VXLAN design must account for underlay reachability, overlay control-plane scalability, tenant segmentation, MTU, endpoint mobility, failure recovery and operational visibility.

The underlay must support the additional encapsulation overhead introduced by VXLAN. Incorrect MTU configuration can cause packet loss or unexpected fragmentation behaviour.

DESIGN CHECK 01
VTEP Reachability

Verify stable IP reachability between participating VTEP loopback addresses.

DESIGN CHECK 02
VNI Consistency

Ensure intended overlay segments and tenant mappings are configured consistently.

DESIGN CHECK 03
Control-Plane Health

Validate BGP EVPN peerings, route advertisements, route-target policy and remote endpoint learning.

Engineering takeaway: EVPN VXLAN is built from three complementary elements: a routed IP underlay, a BGP EVPN overlay control plane and VXLAN data-plane encapsulation. Understanding how these components interact is the foundation for designing and troubleshooting modern data centre fabrics.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 01

EVPN VXLAN Architecture Explorer

Explore a modern leaf-spine data centre fabric. Select endpoints, inspect overlay and underlay forwarding, observe VXLAN encapsulation, and investigate how VTEP reachability and VNI configuration affect traffic delivery.

Interactive EVPN VXLAN Fabric Laboratory

Animated EVPN VXLAN leaf-spine architecture

ROUTED IP UNDERLAY + VXLAN OVERLAY Spine 1 IP Underlay Spine 2 ECMP Fabric Leaf 1 VTEP 10.0.0.11 Leaf 2 VTEP 10.0.0.12 Leaf 3 VTEP 10.0.0.13 Host A VNI 10100 Host B VNI 10100 Host C VNI 10200 Blue Tenant: VNI 10100 | Red Tenant: VNI 10200

Solid endpoint connections represent local attachment. Animated blue links represent routed underlay connectivity. The purple overlay arc illustrates logical VTEP-to-VTEP encapsulation rather than a physical tunnel cable.

Fabric ready.
Source VTEP —
Destination VTEP —
Selected L2VNI —
Forwarding result —

Packet journey and forwarding decisions

Ready

Run a simulation to inspect the packet journey.

EVPN VXLAN engineering console

> EVPN VXLAN ARCHITECTURE EXPLORER > READY
Select endpoints and simulate traffic.
Engineering scope: This is a conceptual forwarding simulator, not a full BGP EVPN implementation. Known unicast assumes the destination MAC/IP binding has already been learned through EVPN Type 2 advertisements. Unknown-unicast and broadcast handling depends on configured flood lists, Type 3 IMET advertisements and the selected BUM replication design. Inter-subnet and inter-tenant routing require additional IRB, VRF and policy configuration and are covered in later labs.
SECTION 02 · LEAF-SPINE UNDERLAY NETWORK DESIGN

Designing the Routed Leaf-Spine Underlay

The IP underlay is the transport foundation of an EVPN VXLAN fabric. Its primary responsibility is to provide reliable Layer 3 connectivity between VTEP loopback addresses using predictable routing, redundant physical paths and fast convergence.

A well-designed underlay must support IP reachability, ECMP, stable loopback addressing, appropriate MTU, failure detection, routing policy and operational observability. Without a healthy underlay, the EVPN VXLAN overlay cannot forward traffic reliably.

2.1 — Leaf-Spine Network Architecture

A conventional two-tier leaf-spine fabric connects each leaf switch to every spine switch. The spines provide transit connectivity between leaf devices, while the leaves connect workloads, servers, firewalls or other network services.

This topology creates multiple equal-cost paths between leaf switches and avoids the need to extend Layer 2 switching across the fabric.

FABRIC ROLE 01
Spine Switches

Provide high-capacity routed transit paths between leaf switches. They do not need to terminate tenant VXLAN tunnels in a conventional leaf-VTEP design.

FABRIC ROLE 02
Leaf Switches

Connect endpoints and commonly act as VTEPs, performing overlay encapsulation, decapsulation and tenant forwarding.

FABRIC ROLE 03
IP Underlay

Carries routed traffic between VTEP loopbacks using IP forwarding and equal-cost paths.

2.2 — Interactive Leaf-Spine Underlay Explorer

Select a network condition to inspect routing, ECMP path selection, spine failure recovery and MTU requirements.

Routed Fabric Architecture and Failure Explorer

UNDERLAY ENGINEERING
SPINE LAYER · IP TRANSIT SPINE 1 Transit Router SPINE 2 Transit Router LEAF LAYER · VTEP LOOPBACK REACHABILITY LEAF 1 10.0.0.11/32 LEAF 2 10.0.0.12/32 LEAF 3 10.0.0.13/32 LEAF 4 10.0.0.14/32 ROUTED LINKS · ECMP · VTEP LOOPBACKS · REDUNDANCY
IP Routing — VTEP Loopback Reachability

The underlay routing protocol advertises the VTEP loopback prefixes. Each leaf learns routes toward remote VTEPs through the available spine paths.

2.3 — Underlay Routing Protocol Selection

Common routed underlay designs use eBGP, OSPF or IS-IS. The selection depends on operational standards, hardware support, routing policy requirements and desired failure isolation.

Protocol Advantages Considerations
eBGP Explicit policy control, scalable routing domains and familiar BGP operations. ASN planning, neighbor configuration and BGP path selection.
OSPF Well-understood link-state routing with ECMP support. Area design, LSDB scale and convergence tuning.
IS-IS Link-state routing widely used in large routed networks. Operational expertise and platform support.

2.4 — eBGP Underlay Architecture

An eBGP underlay establishes routing sessions across point-to-point leaf-spine links. Each leaf advertises its loopback prefix, and the spines propagate reachability to other leaves.

BGP policy and ASN design must permit the intended routes to propagate. When using a common ASN across leaves, designers must account for AS-path loop prevention and platform-specific design patterns.

ILLUSTRATIVE eBGP UNDERLAY SPINE 1 ASN 65000 SPINE 2 ASN 65001 LEAF 1 ASN 65101 LEAF 2 ASN 65102 LEAF 3 ASN 65103 LEAF 4 ASN 65104 LEAF 1: VTEP loopback: 10.0.0.11/32 LEAF 2: VTEP loopback: 10.0.0.12/32 LEAF 1 -> SPINE 1 : eBGP LEAF 1 -> SPINE 2 : eBGP SPINES PROPAGATE UNDERLAY ROUTES LEAF 1 learns: 10.0.0.12/32 via Spine 1 10.0.0.12/32 via Spine 2 ECMP eligibility depends on routing policy and platform multipath configuration.

2.5 — ECMP and Equal-Cost Forwarding

Equal-Cost Multi-Path, or ECMP, allows a router to install multiple eligible next hops toward the same destination prefix.

In a leaf-spine fabric, ECMP can distribute traffic flows across multiple spine switches. Most deployments use flow-based hashing to reduce packet reordering.

ECMP BENEFIT 01
Path Utilization

Multiple routed paths can carry traffic simultaneously, improving aggregate fabric utilization.

ECMP BENEFIT 02
Resilience

When a path fails, forwarding can continue over remaining eligible next hops after failure detection and convergence.

2.6 — Loopback Addressing and VTEP Reachability

VTEP loopback addresses should remain stable independently of individual physical uplink failures.

The underlay must advertise and resolve these addresses. A healthy BGP EVPN session does not guarantee working VXLAN forwarding if the remote VTEP address is unreachable.

VTEP LOOPBACK ADDRESS PLAN LEAF 1 10.0.0.11/32 LEAF 2 10.0.0.12/32 LEAF 3 10.0.0.13/32 LEAF 4 10.0.0.14/32 UNDERLAY VALIDATION 1. Verify routing adjacency 2. Verify remote loopback route 3. Verify ECMP next hops 4. Test source-specific reachability 5. Verify MTU across fabric paths 6. Inspect drops and interface errors

2.7 — BFD and Failure Detection

Bidirectional Forwarding Detection, or BFD, can provide rapid detection of forwarding path failures when supported and correctly integrated with the routing protocol.

Aggressive detection intervals should be tested against platform capabilities and operational stability. Incorrect tuning can cause unnecessary routing reconvergence.

2.8 — MTU Engineering for VXLAN

VXLAN introduces additional encapsulation headers around the original Ethernet frame. The routed underlay must support the resulting packet size.

For a typical VXLAN packet using outer IPv4 without additional options, the VXLAN, UDP and outer IPv4 headers add approximately 50 bytes beyond the original Ethernet frame. Link-layer overhead must also be considered when planning physical interface MTU.

ILLUSTRATIVE VXLAN OVERHEAD Outer IPv4 header 20 bytes UDP header 8 bytes VXLAN header 8 bytes Outer Ethernet header 14 bytes ------------------------------- Total 50 bytes Additional tags, IPv6 headers, options or other encapsulations can change the required MTU. Always validate the platform's MTU accounting conventions.

2.9 — Failure Domains and Convergence

A resilient fabric should tolerate individual uplink and spine failures without losing all connectivity between healthy leaf switches.

Convergence behaviour depends on failure detection, routing protocol processing, route installation and forwarding-plane updates.

FAILURE 01
Leaf Uplink Failure

Traffic can use surviving paths when routing and forwarding converge correctly.

FAILURE 02
Spine Failure

Remaining spines provide alternate transit paths if the topology retains sufficient connectivity.

FAILURE 03
VTEP Reachability Loss

Overlay forwarding may fail even when the EVPN control plane retains previously learned endpoint routes.

2.10 — Production Underlay Troubleshooting

Symptom Likely domain Investigation
Remote VTEP unreachable Underlay routing Inspect loopback routes, next hops and routing adjacencies.
Traffic loss after link failure Convergence Check BFD, routing updates and forwarding-plane installation.
Large packets fail MTU Validate encapsulated packet size across every fabric path.
Uneven path utilization ECMP hashing Review hash inputs, flow distribution and path eligibility.
Unexpected route rejection BGP policy Inspect AS paths, import/export policy and prefix filters.

Engineering takeaway: The routed underlay is the foundation of every EVPN VXLAN fabric. Stable VTEP reachability, predictable ECMP, appropriate MTU, resilient topology and tested convergence behaviour are essential before overlay troubleshooting begins.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 02

Leaf-Spine Fabric Topology Simulator

Investigate the routed underlay supporting EVPN VXLAN. Compare routing protocols, evaluate ECMP path diversity, inject physical and control-plane failures, and verify whether the fabric can transport encapsulated VXLAN packets between VTEP loopbacks.

Interactive IP Fabric Engineering Laboratory

Live routed leaf-spine topology

L3 UNDERLAY / ECMP / VTEP LOOPBACK REACHABILITY Spine 1 AS 65000 Spine 2 AS 65000 Leaf 1 10.0.0.11/32 Leaf 2 10.0.0.12/32 Leaf 3 10.0.0.13/32 Each leaf connects to both spines using routed links
Underlay ready.
Available ECMP paths —
Routing adjacency state —
Encapsulated IP packet size —
VTEP reachability —

Underlay forwarding table

Next hop Path State
Run calculation.

This is a simplified ECMP forwarding model. Real eBGP multipath requires appropriate policy and multipath configuration. IGP and BGP path selection are not interchangeable.

Underlay engineering console

> LEAF-SPINE UNDERLAY LAB > READY
Run the simulation to evaluate the fabric.
MTU calculation: The lab treats the selected inner Ethernet frame size as including its 14-byte Ethernet header, excluding FCS. VXLAN adds 8 bytes, UDP adds 8, outer IPv4 adds 20 or outer IPv6 adds 40 bytes. The calculated outer IP packet size therefore adds 36 bytes for IPv4 or 56 bytes for IPv6. Underlay interface MTU is compared against the resulting outer IP packet size. Actual encapsulation and hardware MTU behavior may vary with VLAN tags, platform overhead and configuration.
SECTION 03 · BGP EVPN CONTROL PLANE ARCHITECTURE

Understanding the BGP EVPN Control Plane

BGP EVPN provides the distributed control plane that allows VTEPs to exchange overlay reachability information across a routed data centre fabric. Rather than relying exclusively on data-plane flooding to discover remote endpoints, EVPN uses MP-BGP to advertise MAC addresses, IP addresses, Ethernet segment information and IP prefixes.

The EVPN control plane separates endpoint discovery, tenant routing policy and overlay signalling from the physical IP underlay. This enables scalable Layer 2 extension and integrated Layer 3 forwarding across distributed VTEPs.

3.1 — Why EVPN Uses MP-BGP

Multiprotocol BGP extends conventional BGP by supporting multiple address families. EVPN is represented using the L2VPN EVPN address family.

Instead of advertising only conventional IP unicast prefixes, BGP EVPN distributes route types that describe Ethernet and overlay reachability information.

CONTROL PLANE 01
Endpoint Discovery

Remote VTEPs learn MAC/IP reachability through EVPN advertisements.

CONTROL PLANE 02
Tenant Segmentation

Route targets control the import and export of EVPN routes between overlay routing contexts.

CONTROL PLANE 03
Scalable Distribution

Route reflectors can reduce the number of required iBGP sessions between EVPN speakers.

3.2 — Interactive BGP EVPN Control-Plane Explorer

Select a control-plane operation to visualize EVPN route advertisement, route reflection, route-target filtering and VTEP next-hop resolution.

BGP EVPN Route Distribution Explorer

MP-BGP EVPN
BGP EVPN CONTROL PLANE EVPN ROUTE REFLECTOR MP-BGP L2VPN EVPN EVPN VTEP CLIENTS LEAF 1 10.0.0.11 LEAF 2 10.0.0.12 LEAF 3 10.0.0.13 LEAF 4 10.0.0.14 EVPN ROUTES · ROUTE TARGETS · REMOTE VTEP NEXT HOPS
EVPN MAC Advertisement

Leaf 1 advertises local MAC/IP reachability through BGP EVPN. The route reflector distributes the route to eligible EVPN clients, subject to BGP policy.

3.3 — EVPN Route Distinguishers

A Route Distinguisher, or RD, makes otherwise identical EVPN route identifiers unique within the BGP control plane.

Different VTEPs may advertise similar MAC/IP information. Unique RDs allow BGP to distinguish their advertisements as separate routes.

EVPN ROUTE DISTINGUISHER EXAMPLE LEAF 1 RD: 10.0.0.11:10010 VNI: 10010 LEAF 2 RD: 10.0.0.12:10010 VNI: 10010 RD PURPOSE: Distinguish otherwise overlapping EVPN NLRI. IMPORTANT: RD does not determine tenant route import policy.

3.4 — Route Targets and Tenant Import Policy

A Route Target, or RT, is an extended community used to control route import and export between EVPN routing contexts.

A VTEP imports eligible EVPN routes according to its configured route-target policy. Incorrect route-target configuration can prevent endpoints from learning remote reachability.

ROUTE TARGET 01
Export RT

Applied to advertised EVPN routes to identify their intended import membership.

ROUTE TARGET 02
Import RT

Determines which EVPN routes a receiving routing context is eligible to import.

TENANT ROUTE-TARGET EXAMPLE TENANT BLUE L2VNI: 10010 Export RT: 65000:10010 Import RT: 65000:10010 TENANT GREEN L2VNI: 20020 Export RT: 65000:20020 Import RT: 65000:20020 RESULT: Routes are imported into matching overlay contexts according to RT policy. RD = Route uniqueness RT = Route import/export policy

3.5 — EVPN Route Reflector Architecture

A full mesh of iBGP EVPN sessions becomes increasingly difficult to manage as the number of VTEPs grows.

Route reflectors reduce this session requirement by reflecting eligible routes between iBGP clients.

In EVPN VXLAN deployments, route reflectors commonly operate as control-plane distribution points and do not need to participate in tenant VXLAN data-plane forwarding.

BGP EVPN ROUTE REFLECTION EVPN RR / | \ / | \ LEAF1 LEAF2 LEAF3 LEAF1: Originates EVPN Type 2 route RR: Receives eligible route Applies reflection rules Advertises to clients LEAF2: Receives reflected route Evaluates import RT Resolves remote VTEP next hop Installs eligible reachability DATA PLANE: LEAF1 <---- VXLAN ----> LEAF2 The RR is not necessarily in the VXLAN forwarding path.

3.6 — EVPN MAC/IP Advertisement Routes

EVPN Route Type 2 advertises MAC reachability and may include an associated IP address. This enables remote VTEPs to learn endpoint information through the control plane.

Depending on the EVPN service and implementation, these routes support remote MAC learning, MAC mobility handling and host reachability in integrated routing-and-bridging designs.

ILLUSTRATIVE EVPN TYPE 2 ROUTE Route Type: 2 RD: 10.0.0.11:10010 MAC: 00:11:22:33:44:55 IP: 192.168.10.11 BGP Next Hop: 10.0.0.11 Route Target: 65000:10010 VXLAN VNI: 10010 Meaning: Endpoint reachability is advertised from Leaf 1. Exact displayed attributes depend on the EVPN encoding and platform implementation.

3.7 — BGP Next-Hop Resolution

Receiving an EVPN route does not automatically guarantee usable data-plane forwarding.

The receiving VTEP must resolve the route's next hop through the IP underlay. If the remote VTEP address is unreachable, overlay forwarding can fail.

VALIDATION 01
EVPN Route Received

Confirm that the relevant EVPN route is present in the BGP table.

VALIDATION 02
Route Imported

Verify the route target matches the intended tenant routing context.

VALIDATION 03
Next Hop Resolved

Confirm that the underlay can reach the remote VTEP address.

3.8 — EVPN Control-Plane Versus Data-Plane Learning

EVPN distributes reachability information through the control plane, reducing the need to discover remote endpoints solely through flooding.

However, the behaviour of unknown unicast, broadcast and multicast traffic depends on the deployed EVPN service, VNI configuration and BUM replication design.

Function Control plane Data plane
Endpoint discovery BGP EVPN route advertisements. Local MAC learning and applicable forwarding behaviour.
Remote reachability EVPN route import and next-hop resolution. VXLAN encapsulation toward remote VTEPs.
Tenant policy Route targets and routing-context configuration. VNI/VRF forwarding and enforced policy.
MAC mobility EVPN mobility signalling where applicable. Updated forwarding toward the current endpoint location.

3.9 — Production BGP EVPN Troubleshooting

Symptom Likely cause Investigation
Remote MAC missing EVPN route not advertised or received. Inspect EVPN BGP sessions and Type 2 routes.
Route received but not imported Route-target mismatch. Compare import/export RT configuration.
EVPN route present but forwarding fails Unresolved VTEP next hop or data-plane issue. Verify underlay reachability, VNI mapping and VXLAN forwarding.
Some VTEPs do not learn routes Route reflection or policy issue. Inspect RR client sessions, reflection rules and outbound policy.
Incorrect tenant route visibility RT policy or VRF association error. Validate tenant segmentation and route import policy.

Engineering takeaway: BGP EVPN is the distributed overlay control plane that enables VTEPs to learn remote endpoint reachability. Route distinguishers provide uniqueness, route targets govern import policy, route reflectors improve session scalability, and underlay next-hop resolution connects the control plane to usable VXLAN forwarding.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 03

BGP EVPN Control Plane Explorer

Examine how MP-BGP EVPN distributes MAC/IP advertisements between leaf VTEPs. Manipulate route reflectors, route targets, route distinguishers and session state to understand control-plane propagation and remote MAC learning.

Interactive BGP EVPN Engineering Laboratory

Animated BGP EVPN control-plane topology

MP-BGP EVPN / AFI 25 / SAFI 70 EVPN Route Reflector AS 65000 / RR Leaf 1 VTEP 10.0.0.11 Leaf 2 VTEP 10.0.0.12 Leaf 3 VTEP 10.0.0.13 EVPN routes propagate through the route reflector
Control plane ready.
Originating VTEP —
Route distinguisher —
Route target —
Import decision —

EVPN route advertisement and import table

Attribute Value
Run advertisement.

BGP EVPN engineering console

> BGP EVPN CONTROL PLANE EXPLORER > READY
Advertise an EVPN route to inspect propagation.
Engineering distinction: A route distinguisher makes an EVPN NLRI unique; it does not by itself determine which tenant imports the route. Route targets control import policy. The BGP EVPN next hop identifies the advertising VTEP, and usable forwarding additionally depends on underlay reachability and correct VNI/VRF mapping. Route reflection does not automatically guarantee successful forwarding.
SECTION 04 · EVPN ROUTE TYPES

EVPN Route Types: How the Fabric Advertises Reachability

EVPN uses multiple BGP route types to communicate different kinds of overlay information. Each route type serves a particular architectural role, from endpoint discovery and BUM replication to multihoming and inter-subnet routing.

Understanding these routes is essential when troubleshooting missing MAC addresses, incorrect tenant forwarding, multicast replication, Ethernet segment failures and unavailable IP prefixes.

4.1 — The Five Core EVPN Route Types

Type Route name Primary purpose
1 Ethernet Auto-Discovery Ethernet segment discovery, mass withdrawal and multihoming-related signalling.
2 MAC/IP Advertisement Advertise endpoint MAC addresses and optionally associated IP addresses.
3 Inclusive Multicast Ethernet Tag Discover participants for BUM traffic distribution within an EVPN service.
4 Ethernet Segment Discover multihoming peers and support designated-forwarder election.
5 IP Prefix Advertise IP prefixes for EVPN-based routed overlay services.

4.2 — Interactive EVPN Route-Type Explorer

Select any EVPN route type. The architecture diagram, example route information and explanation will update to show its role.

EVPN Route-Type Architecture Explorer

BGP L2VPN EVPN
EVPN CONTROL PLANE EVPN ROUTE REFLECTOR MP-BGP ROUTE DISTRIBUTION LEAF VTEPS LEAF 1 VTEP 10.0.0.11 LEAF 2 VTEP 10.0.0.12 LEAF 3 VTEP 10.0.0.13 LEAF 4 VTEP 10.0.0.14 ROUTE TYPE · RD · RT · NEXT HOP · EVPN NLRI
Type 1 — Ethernet Auto-Discovery

Ethernet Auto-Discovery routes support EVPN multihoming, including per-ES and per-EVI advertisements. They are important for mass withdrawal and aliasing-related procedures.

4.3 — Type 1: Ethernet Auto-Discovery Routes

Type 1 routes advertise Ethernet Auto-Discovery information and play an important role in EVPN multihoming.

Per-Ethernet-Segment and per-EVI advertisements support procedures such as mass withdrawal and aliasing, depending on the EVPN service and deployment.

TYPE 1 · OPERATIONAL EXAMPLE
Multihomed Ethernet Segment Failure

When a PE loses connectivity to an Ethernet segment, withdrawal of relevant Ethernet Auto-Discovery reachability can help remote devices invalidate affected forwarding paths without waiting for individual MAC routes to age out.

4.4 — Type 2: MAC/IP Advertisement Routes

Type 2 routes are central to EVPN endpoint discovery. They advertise a MAC address and may include an associated IP address.

A receiving VTEP can use this information to determine where remote endpoints are reachable.

ILLUSTRATIVE TYPE 2 ROUTE Route Type: 2 RD: 10.0.0.11:10010 MAC: 00:11:22:33:44:55 IP: 192.168.10.11 RT: 65000:10010 Next Hop: 10.0.0.11 Meaning: MAC/IP reachability advertised by Leaf 1. Additional EVPN attributes and labels/VNI encoding depend on the service.

4.5 — Type 3: Inclusive Multicast Ethernet Tag Routes

Type 3 routes identify participants in an EVPN broadcast domain and support the construction of BUM traffic distribution state.

BUM stands for Broadcast, Unknown Unicast and Multicast. Depending on the deployment, the VXLAN fabric may use ingress replication or an appropriate multicast transport mechanism.

TYPE 3 · OPERATIONAL EXAMPLE
Ingress Replication

A VTEP can learn the other participating VTEPs for a broadcast domain and replicate relevant BUM frames toward the required remote tunnel endpoints.

4.6 — Type 4: Ethernet Segment Routes

Type 4 routes are used in EVPN multihoming to discover PEs attached to the same Ethernet segment.

They support designated-forwarder election procedures, which coordinate BUM forwarding toward multihomed Ethernet segments.

ILLUSTRATIVE ETHERNET SEGMENT CE / SERVER | ETHERNET SEGMENT / \ / \ LEAF 1 LEAF 2 \ / EVPN FABRIC Shared ESI identifies the Ethernet segment. Type 4 routes help discover participating PEs for DF election.

4.7 — Type 5: IP Prefix Advertisement Routes

Type 5 routes advertise IP prefixes through BGP EVPN and are important in routed overlay designs.

They allow IP prefix reachability to be distributed independently of advertising every endpoint as an individual MAC/IP route.

ILLUSTRATIVE TYPE 5 ROUTE Route Type: 5 RD: 10.0.0.11:50000 IP Prefix: 172.20.40.0/24 Route Target: 65000:50000 BGP Next Hop: 10.0.0.11 Purpose: Advertise routed overlay IP reachability. Actual next-hop resolution may also involve an EVPN overlay index.

4.8 — Route-Type Dependencies

EVPN route types are not interchangeable. They provide complementary control-plane information for different service models.

LAYER 2 CONNECTIVITY
Type 2 + Type 3

Type 2 routes advertise endpoint reachability. Type 3 routes support BUM distribution membership and replication state.

MULTIHOMING
Type 1 + Type 4

Type 1 routes support Ethernet Auto-Discovery procedures, while Type 4 routes support Ethernet segment peer discovery and DF election.

LAYER 3 OVERLAY
Type 5

Type 5 routes advertise IP prefixes for routed EVPN services, subject to the configured resolution and forwarding model.

CONTROL-PLANE POLICY
RD + RT + Next Hop

Route distinguishers, route targets and next-hop resolution determine uniqueness, import eligibility and forwarding usability.

4.9 — Production Route-Type Troubleshooting

Symptom Route type to inspect Engineering investigation
Remote MAC missing Type 2 Verify MAC advertisement, RT import and remote VTEP reachability.
BUM replication incomplete Type 3 Inspect IMET routes and replication membership.
Multihoming convergence issue Type 1 Inspect Ethernet AD routes and withdrawals.
Unexpected DF behaviour Type 4 Inspect Ethernet segment routes, ESI and DF election parameters.
Remote IP prefix missing Type 5 Verify prefix advertisement, RT import and next-hop resolution.

Engineering takeaway: EVPN Route Types 1–5 describe different parts of the overlay control plane. Type 1 supports Ethernet Auto-Discovery, Type 2 advertises MAC/IP reachability, Type 3 supports BUM distribution, Type 4 supports Ethernet segment discovery, and Type 5 advertises IP prefixes.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 04

EVPN Route Type Investigation Lab

Decode EVPN control-plane advertisements, investigate route-target import policy and observe how different route types support MAC reachability, BUM replication, IP prefix routing and Ethernet-segment multihoming.

BGP EVPN Route Analysis Laboratory

Animated EVPN route-distribution architecture

BGP EVPN ROUTE DISTRIBUTION / AFI 25 SAFI 70 EVPN Route Reflector MP-BGP / Route Distribution Leaf 1 10.0.0.11 Leaf 2 10.0.0.12 Leaf 3 10.0.0.13 Route propagation → RT import → EVPN forwarding state
EVPN route investigation ready.
EVPN route type —
Originating VTEP —
Route target —
Import decision —

Decoded EVPN route attributes

FieldDecoded value
Run a route investigation.

EVPN engineering console

> EVPN ROUTE INVESTIGATION > READY
Select an EVPN route type and investigate its behavior.
Engineering context: EVPN Type 1 supports Ethernet Auto-Discovery, Type 2 advertises MAC/IP reachability, Type 3 advertises inclusive multicast membership, Type 4 supports Ethernet-segment discovery and designated-forwarder election, and Type 5 advertises IP prefixes. Exact import rules, extended communities, NLRI fields and forwarding behavior differ by route type and deployment model. This laboratory illustrates the primary functions rather than implementing a complete RFC-compliant EVPN BGP stack.
SECTION 05 · VXLAN ENCAPSULATION AND VTEP FORWARDING

How VXLAN Packets Travel Across the Data Centre Fabric

VXLAN enables Ethernet frames to traverse a routed IP fabric by encapsulating them inside UDP/IP packets. The ingress VTEP adds the VXLAN encapsulation, the underlay forwards the outer IP packet, and the egress VTEP removes the encapsulation before delivering the original frame.

Understanding this forwarding process is critical for diagnosing packet loss, incorrect VNI mapping, remote MAC reachability problems, MTU failures, BUM replication issues and ECMP path behaviour.

5.1 — VXLAN Encapsulation Architecture

VXLAN uses a 24-bit VNI to identify the logical overlay segment associated with the encapsulated Ethernet frame.

The outer IP header identifies the source and destination VTEPs, while the UDP destination port is conventionally 4789.

VXLAN ENCAPSULATED FRAME +--------------------------+ | Outer Ethernet Header | +--------------------------+ | Outer IP Header | | Source: Ingress VTEP | | Destination: Egress VTEP | +--------------------------+ | UDP Header | | Destination Port: 4789 | +--------------------------+ | VXLAN Header | | VNI: 10010 | +--------------------------+ | Original Ethernet Frame | | Original IP / Payload | +--------------------------+ | Outer Ethernet FCS | +--------------------------+ The outer FCS is generated for the physical Ethernet link and is not part of the IP VXLAN payload.

5.2 — Interactive VXLAN Packet Journey Explorer

Select a forwarding stage to inspect how an Ethernet frame becomes a VXLAN packet, crosses the routed underlay, and is delivered to the remote host.

VXLAN Packet Journey and BUM Replication

DATA PLANE EXPLORER
VXLAN DATA-PLANE FORWARDING SPINE FABRIC ROUTED IP / ECMP LEAF 1 VTEP 10.0.0.11 LEAF 2 VTEP 10.0.0.12 LEAF 3 VTEP 10.0.0.13 LEAF 4 VTEP 10.0.0.14 HOST A VNI 10010 HOST B VNI 10010 INNER ETHERNET FRAME · VXLAN HEADER · OUTER IP TRANSPORT
Stage 1 — Ingress Ethernet Frame

Host A sends an Ethernet frame toward Host B. Leaf 1 receives the frame and determines the corresponding local VLAN and Layer 2 VNI.

5.3 — VTEP Forwarding Decisions

Before encapsulating a frame, the ingress VTEP must determine whether the destination is local or remote.

For known remote unicast destinations, the VTEP uses its overlay forwarding information to identify the destination VTEP and appropriate VNI.

DECISION 01
Local Forwarding

If the destination is locally attached, the switch forwards without remote VXLAN encapsulation.

DECISION 02
Remote Unicast

A known remote destination is encapsulated toward the learned remote VTEP.

DECISION 03
BUM Traffic

Broadcast, unknown unicast and multicast frames follow the configured replication and flooding rules.

5.4 — Remote MAC Learning

In a BGP EVPN deployment, remote MAC reachability can be learned through EVPN Type 2 route advertisements.

The receiving VTEP associates the remote MAC with the appropriate overlay forwarding context and remote VTEP.

REMOTE MAC FORWARDING EXAMPLE VNI: 10010 MAC ADDRESS DESTINATION ----------------------------------- 00:11:22:33:44:55 Local Host A 00:aa:bb:cc:dd:ee VTEP 10.0.0.12 When Host A sends traffic to 00:aa:bb:cc:dd:ee: Ingress VTEP: Leaf 1 Remote VTEP: 10.0.0.12 Encapsulation: VXLAN VNI 10010

5.5 — BUM Replication and Flooding

BUM traffic requires special handling because it may need to reach multiple participating VTEPs within a broadcast domain.

EVPN Type 3 routes can advertise replication membership for the corresponding EVPN service.

In ingress-replication deployments, the originating VTEP creates separate encapsulated copies for relevant remote VTEPs.

ILLUSTRATIVE BUM REPLICATION VNI 10010 INGRESS VTEP: 10.0.0.11 REMOTE PARTICIPANTS: 10.0.0.12 10.0.0.13 10.0.0.14 BUM FRAME | +--> VXLAN copy to .12 | +--> VXLAN copy to .13 | +--> VXLAN copy to .14 Only VTEPs participating in the relevant service should receive its BUM replication traffic.

5.6 — ECMP Hashing and UDP Source Ports

VXLAN uses UDP encapsulation, which allows the outer UDP source port to provide entropy for ECMP hashing.

This helps the underlay distribute different encapsulated flows across multiple eligible paths.

Individual flows are typically kept on a consistent path to reduce packet reordering, although exact behaviour depends on the forwarding platform and hash configuration.

HASH INPUT 01
Outer IP Addresses

Source and destination VTEP IP addresses identify tunnel endpoints.

HASH INPUT 02
UDP Source Port

An entropy-derived UDP source port helps differentiate encapsulated flows.

HASH INPUT 03
Platform Hash Policy

Hardware determines which header fields influence ECMP path selection.

5.7 — VXLAN MTU and Encapsulation Overhead

The underlay must support the original frame plus the additional VXLAN, UDP and outer IP headers.

A conventional outer IPv4 VXLAN encapsulation adds approximately 50 bytes of overhead when counting the outer Ethernet header, without additional VLAN tags or optional headers.

VXLAN OVERHEAD EXAMPLE Outer Ethernet 14 bytes Outer IPv4 20 bytes UDP 8 bytes VXLAN 8 bytes ----------------------------- Total 50 bytes Additional overhead: Outer VLAN tags Outer IPv6 Other encapsulations Check the platform's MTU accounting model.

5.8 — Packet Capture and Forwarding Validation

Packet captures provide valuable evidence when diagnosing VXLAN forwarding failures.

A capture taken on an underlay interface can reveal the outer VTEP addresses, UDP destination port and VNI.

WIRESHARK / PACKET CAPTURE CHECKS 1. Outer source VTEP IP 2. Outer destination VTEP IP 3. UDP destination port 4789 4. VXLAN VNI value 5. Inner Ethernet source MAC 6. Inner Ethernet destination MAC 7. Inner VLAN/traffic context 8. Packet length and fragmentation 9. Replication destination list Useful capture filter: udp port 4789

5.9 — Production VXLAN Troubleshooting

Symptom Likely issue Investigation
Remote host unreachable Missing remote MAC or incorrect VTEP mapping. Inspect EVPN Type 2 routes and overlay forwarding entries.
VXLAN packets dropped MTU, filtering or unsupported encapsulation. Validate packet size, ACLs and VTEP interface state.
Broadcast traffic missing BUM replication membership issue. Inspect EVPN Type 3 routes and replication lists.
Unexpected VNI VLAN-to-VNI mapping error. Compare ingress and egress overlay configuration.
Uneven fabric traffic ECMP hash behaviour. Inspect UDP source-port entropy and hardware hash policy.

Engineering takeaway: VXLAN forwarding depends on correct endpoint learning, VNI mapping, remote VTEP resolution, encapsulation, underlay transport and decapsulation. When troubleshooting, inspect both the inner tenant frame and the outer VXLAN transport packet.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 05

VXLAN Packet Encapsulation Simulator

Construct and inspect a VXLAN-encapsulated Ethernet frame. Explore VTEP selection, VXLAN headers, UDP port 4789, outer IP forwarding, BUM replication and remote decapsulation.

Interactive VXLAN Packet Engineering Laboratory

Animated VXLAN packet forwarding topology

VXLAN DATA PLANE / UDP 4789 / VTEP FORWARDING IP Spine Fabric Outer IP Routing Ingress VTEP 10.0.0.11 Egress VTEP 10.0.0.12
Packet simulation ready.
Inner Ethernet frame —
VXLAN overhead —
Outer IP packet —
Forwarding result —

VXLAN packet header inspector

Outer IP Header Source → Destination
UDP Header Destination Port 4789
VXLAN Header VNI 10100
Inner Ethernet Frame Source MAC → Destination MAC
Payload Original workload data

The outer IP header is routed through the underlay. The inner Ethernet frame remains encapsulated until the destination VTEP decapsulates the packet.

VXLAN forwarding console

> VXLAN PACKET SIMULATOR > READY
Configure a packet and run the simulation.
Packet-size model: The selected inner Ethernet frame includes its 14-byte Ethernet header and excludes FCS. VXLAN adds an 8-byte VXLAN header and an 8-byte UDP header. The outer IP header adds 20 bytes for IPv4 or 40 bytes for IPv6. The model compares the resulting outer IP packet size with the underlay IP MTU. Actual deployments must account for VLAN tags, outer Ethernet framing and platform-specific MTU behavior.
SECTION 06 · ANYCAST GATEWAYS AND INTEGRATED ROUTING AND BRIDGING

Distributed Anycast Gateways and EVPN VXLAN IRB

EVPN VXLAN supports distributed Layer 3 gateways, allowing multiple leaf switches to provide the same default gateway IP address and virtual gateway MAC within a tenant subnet.

This architecture enables hosts to use a local first-hop gateway even when workloads are distributed across different VTEPs. Integrated Routing and Bridging, or IRB, combines Layer 2 overlay forwarding with Layer 3 routing between tenant subnets.

6.1 — Distributed Anycast Gateway Fundamentals

In a distributed anycast gateway design, multiple VTEPs provide equivalent gateway addresses for the same tenant subnet.

A host sends traffic to its default gateway, and the locally attached VTEP can perform the Layer 3 routing decision.

DISTRIBUTED ANYCAST GATEWAY TENANT BLUE Subnet: 192.168.10.0/24 Gateway IP: 192.168.10.1 Virtual Gateway MAC: 00:00:5e:00:01:10 (illustrative value) LEAF 1: Gateway 192.168.10.1 Virtual MAC configured LEAF 2: Gateway 192.168.10.1 Same virtual MAC configured HOSTS: Use 192.168.10.1 as their default gateway The virtual MAC is a configured design value; platform conventions vary.
ADVANTAGE 01
Local First Hop

Hosts can reach a default gateway on their directly attached leaf.

ADVANTAGE 02
Distributed Routing

Layer 3 forwarding can occur at multiple VTEPs rather than a single central gateway.

ADVANTAGE 03
Workload Mobility

Consistent gateway addressing can simplify host mobility within supported overlay designs.

6.2 — Interactive Anycast Gateway and IRB Explorer

Select a forwarding model to inspect same-subnet bridging, symmetric IRB, asymmetric IRB and a gateway-related failure scenario.

Anycast Gateway and IRB Architecture Explorer

DISTRIBUTED L3 GATEWAY
EVPN VXLAN IRB FABRIC ROUTED UNDERLAY IP / ECMP TRANSPORT LEAF 1 / VTEP Anycast Gateway 10.0.0.11 LEAF 2 / VTEP Anycast Gateway 10.0.0.12 HOST A 192.168.10.11 / VNI 10010 HOST B 192.168.20.22 / VNI 20020
Same-Subnet Layer 2 Forwarding

For same-subnet forwarding, traffic can remain in the Layer 2 overlay without requiring inter-subnet routing. The VTEPs bridge the tenant Ethernet frame across the corresponding L2VNI.

6.3 — Layer 2 VNIs and Layer 3 VNIs

A Layer 2 VNI identifies a logical Ethernet segment, while a Layer 3 VNI is commonly associated with a tenant VRF in symmetric IRB architectures.

L2VNI
Layer 2 Overlay Segment

Example: VNI 10010 for subnet 192.168.10.0/24. Supports tenant Ethernet forwarding.

L3VNI
Tenant Routed Context

Example: VNI 50000 associated with VRF BLUE. Supports routed overlay forwarding in a symmetric IRB design.

6.4 — Symmetric IRB Architecture

In symmetric IRB, the ingress VTEP routes the packet into the tenant routed overlay, and the egress VTEP routes it from that overlay toward the destination subnet.

The routed overlay commonly uses a Layer 3 VNI associated with the tenant VRF.

SYMMETRIC IRB TRAFFIC FLOW HOST A 192.168.10.11 | v LEAF 1 Ingress L2VNI: 10010 Local anycast gateway Route in VRF BLUE | v VXLAN ROUTED OVERLAY L3VNI: 50000 | v LEAF 2 VRF BLUE Route toward destination Egress L2VNI: 20020 | v HOST B 192.168.20.22 Both ingress and egress VTEPs participate in the routed forwarding process.

6.5 — Asymmetric IRB Architecture

In asymmetric IRB, the ingress VTEP performs inter-subnet routing and then bridges the packet across the destination Layer 2 VNI.

This design generally requires the ingress VTEP to have the relevant destination subnet's L2VNI and associated forwarding context.

ASYMMETRIC IRB TRAFFIC FLOW HOST A 192.168.10.11 | v LEAF 1 Ingress L2VNI: 10010 Route in tenant VRF Select destination subnet | v VXLAN L2 OVERLAY Destination L2VNI: 20020 | v LEAF 2 Bridge within VNI 20020 | v HOST B 192.168.20.22 Routing occurs at the ingress VTEP. Remote forwarding uses destination L2VNI.

6.6 — Symmetric Versus Asymmetric IRB

Characteristic Symmetric IRB Asymmetric IRB
Routing model Ingress and egress VTEPs participate in routed forwarding. Ingress VTEP performs inter-subnet routing.
Transit VNI Typically uses a tenant L3VNI. Uses the destination subnet's L2VNI.
Subnet distribution Does not require every leaf to instantiate every tenant L2VNI. Ingress leaf generally needs relevant destination L2VNI state.
Scalability Often preferred for larger distributed routed overlays. Can be practical in smaller or constrained designs.
Operational focus VRF, L3VNI, EVPN routes and next-hop resolution. Destination L2VNI, MAC learning and local routing state.

6.7 — Tenant VRFs and Routing Isolation

Virtual Routing and Forwarding instances provide independent Layer 3 routing tables for different tenants.

A tenant VRF can contain multiple subnets and their corresponding Layer 2 VNIs.

TENANT VRF EXAMPLE VRF BLUE L3VNI: 50000 SUBNET A 192.168.10.0/24 L2VNI: 10010 SUBNET B 192.168.20.0/24 L2VNI: 20020 VRF GREEN L3VNI: 60000 SUBNET C 10.30.30.0/24 L2VNI: 30030 VRFs provide separate routing contexts. Inter-VRF connectivity requires explicit design and policy.

6.8 — EVPN Routes Used in IRB

EVPN Type 2 routes can advertise host MAC/IP reachability. Type 5 routes can advertise routed IP prefixes.

The exact EVPN route combination and next-hop resolution model depend on the selected IRB architecture and platform implementation.

ROUTE TYPE 2
Host Reachability

Advertises MAC/IP endpoint information used by relevant EVPN forwarding procedures.

ROUTE TYPE 5
IP Prefix Reachability

Advertises routed prefixes for supported EVPN Layer 3 services.

NEXT-HOP RESOLUTION
Remote VTEP

Requires usable transport reachability and correct overlay forwarding state.

6.9 — Production IRB Troubleshooting

Symptom Likely issue Investigation
Same-subnet traffic works, inter-subnet fails VRF routing, IRB configuration or L3VNI issue. Inspect tenant routing table, gateway interfaces and L3VNI mapping.
Host cannot reach default gateway Anycast gateway or local VLAN/VNI mapping issue. Verify gateway IP/MAC, SVI state and local host attachment.
Traffic fails between remote subnets Missing EVPN reachability or unresolved next hop. Inspect Type 2/5 routes, route targets and VTEP reachability.
Asymmetric IRB fails for selected subnets Missing destination L2VNI or MAC state. Verify required L2VNIs and destination forwarding entries.
Unexpected cross-tenant connectivity Incorrect VRF association or route leaking. Review tenant VRFs, route targets and security policy.

Engineering takeaway: Distributed anycast gateways place the default gateway close to workloads, while IRB provides routing between tenant subnets. Symmetric IRB commonly uses a tenant L3VNI for routed overlay transport, whereas asymmetric IRB routes at ingress and bridges across the destination L2VNI.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 06

Anycast Gateway and IRB Traffic Flow Simulator

Investigate distributed anycast gateways, symmetric and asymmetric IRB, tenant VRFs, L2VNIs and L3VNIs. Follow a packet from a source workload through ingress routing, VXLAN encapsulation, remote decapsulation and destination delivery.

EVPN VXLAN Distributed Gateway Laboratory

Animated distributed anycast gateway architecture

DISTRIBUTED ANYCAST GATEWAYS / SYMMETRIC + ASYMMETRIC IRB Routed Spine Fabric IP Underlay / ECMP Leaf 1 Anycast GW / VTEP Leaf 2 Anycast GW / VTEP Leaf 3 Anycast GW / VTEP Host A 192.168.10.11 Host B 192.168.20.22 Host C 192.168.30.33
IRB fabric ready.
Forwarding mode —
Overlay VNI —
Tenant VRF —
Forwarding result —

IRB packet journey and forwarding table

StageForwarding operation
Run the IRB simulation.

Anycast gateway engineering console

> EVPN VXLAN IRB SIMULATOR > READY
Select workloads and simulate traffic.
Architecture distinction: Symmetric IRB performs routing at the ingress and egress VTEPs and typically transports inter-subnet traffic using an L3VNI associated with the tenant VRF. Asymmetric IRB performs routing at the ingress VTEP and transports traffic in the destination L2VNI, requiring the relevant destination bridge domain on the ingress leaf. Type 2 MAC/IP reachability and Type 5 IP prefix advertisements can support different forwarding designs. This simulator models the essential decision points rather than implementing every EVPN IRB procedure.
SECTION 07 · MULTI-TENANCY, SECURITY AND EVPN MULTIHOMING

Tenant Isolation, Security and Resilient EVPN Multihoming

EVPN VXLAN enables multiple logical networks to share the same physical leaf-spine infrastructure while maintaining separate Layer 2 and Layer 3 forwarding contexts.

Multi-tenancy is achieved through combinations of VNIs, VRFs, route-target policy and explicit security controls. EVPN multihoming adds resilience by allowing a device or Ethernet segment to connect to multiple provider edge devices.

7.1 — Multi-Tenant EVPN VXLAN Architecture

A tenant may have one or more Layer 2 VNIs associated with its subnets and a Layer 3 VNI associated with its routed VRF in symmetric IRB designs.

Separate VRFs maintain independent routing tables. Route targets control the import of EVPN reachability into the intended overlay contexts.

MULTI-TENANT EVPN EXAMPLE TENANT BLUE VRF: BLUE L3VNI: 50000 Subnet A: 192.168.10.0/24 L2VNI: 10010 Subnet B: 192.168.20.0/24 L2VNI: 20020 TENANT GREEN VRF: GREEN L3VNI: 60000 Subnet C: 10.30.30.0/24 L2VNI: 30030 Shared physical fabric Separate tenant routing and forwarding contexts.

7.2 — Interactive Multi-Tenant and Multihoming Explorer

Select a scenario to inspect tenant isolation, all-active multihoming, single-active forwarding and Ethernet segment failure recovery.

EVPN Tenant Isolation and Multihoming Explorer

EVPN SECURITY + RESILIENCE
EVPN VXLAN FABRIC EVPN CONTROL PLANE ROUTE DISTRIBUTION LEAF 1 VTEP 10.0.0.11 Tenant VRFs LEAF 2 VTEP 10.0.0.12 Tenant VRFs DUAL-HOMED SERVER / CE SHARED ETHERNET SEGMENT
Tenant Isolation

Separate tenant VRFs, VNIs and route-target policies restrict which overlay routes are imported into each forwarding context.

7.3 — Route Targets and Tenant Isolation

Route targets determine which EVPN routes are imported into particular tenant contexts.

Incorrect route-target configuration can create unintended route visibility or prevent legitimate remote reachability.

TENANT BLUE
RT 65000:50000

Imported by the intended BLUE routed overlay contexts according to their configured policy.

TENANT GREEN
RT 65000:60000

Imported by the intended GREEN routed overlay contexts according to their configured policy.

7.4 — Security Boundaries in EVPN VXLAN

VRFs and VNIs provide forwarding separation, but they should not be treated as a complete security architecture.

Production deployments often require additional controls such as ACLs, firewalls, segmentation policy, route-leaking governance and management-plane protection.

SECURITY 01
VRF Isolation

Maintain separate routing tables for tenants.

SECURITY 02
Policy Enforcement

Apply appropriate ACLs, firewalls and inter-tenant traffic controls.

SECURITY 03
Control-Plane Protection

Protect BGP sessions, management access and route advertisement policy.

7.5 — Ethernet Segment Identifiers

EVPN multihoming uses an Ethernet Segment Identifier, or ESI, to identify an Ethernet segment attached to multiple EVPN provider edge devices.

Participating PEs advertise the relevant Ethernet segment information using EVPN control-plane routes.

ETHERNET SEGMENT EXAMPLE DUAL-HOMED CE / \ / \ LEAF 1 LEAF 2 \ / EVPN FABRIC Ethernet Segment Identifier: 0000:0000:0000:0000:0001 EVPN ROUTES: Type 1: Ethernet AD Type 4: Ethernet Segment ESI identifies the shared Ethernet segment.

7.6 — All-Active EVPN Multihoming

In all-active multihoming, multiple PEs attached to an Ethernet segment can participate in forwarding, subject to the EVPN service and forwarding procedures.

EVPN multihoming mechanisms help coordinate remote reachability, aliasing and loop prevention.

ALL-ACTIVE DESIGN
Redundant Forwarding Paths

Multiple participating PEs can provide forwarding paths for the same Ethernet segment. BUM forwarding toward the segment is coordinated through designated- forwarder procedures.

7.7 — Single-Active EVPN Multihoming

In single-active multihoming, only one PE is designated to forward relevant traffic for an Ethernet segment or service at a time, according to the applicable EVPN procedures.

This can simplify forwarding behaviour for certain attached systems, while potentially limiting simultaneous path utilization.

MULTIHOMING COMPARISON ALL-ACTIVE Leaf 1: Participating Leaf 2: Participating SINGLE-ACTIVE Leaf 1: Active Leaf 2: Standby Actual forwarding roles depend on EVPN mode, traffic direction and DF election procedures.

7.8 — Designated Forwarder Election

The Designated Forwarder, or DF, is selected to coordinate forwarding of relevant BUM traffic from the EVPN network toward a multihomed Ethernet segment.

EVPN Type 4 routes support Ethernet segment peer discovery, which contributes to DF election.

DF election algorithms and operational behaviour vary according to supported EVPN standards and platform implementations.

7.9 — Multihoming Failure Recovery

When a participating PE or attachment fails, EVPN control-plane updates and local forwarding changes can redirect traffic toward remaining usable paths.

Recovery depends on failure detection, Ethernet segment signalling, routing convergence and correct forwarding-plane updates.

Failure Expected design behaviour Investigation
One CE-facing link fails Surviving attachment may continue forwarding. Check ESI membership, interface state and EVPN withdrawals.
One PE fails Remaining PE may preserve service connectivity. Inspect Type 1/4 routes and convergence.
Unexpected duplicate BUM traffic DF procedures should prevent inappropriate duplicate delivery. Inspect DF election state and Ethernet segment configuration.
Remote traffic blackholes Forwarding should avoid unavailable PE paths after convergence. Check aliasing, mass withdrawal and remote forwarding state.
Tenant route leak Unauthorized cross-tenant reachability should be prevented by policy. Review RTs, VRFs, route leaking and security controls.

Engineering takeaway: EVPN VXLAN multi-tenancy combines VNIs, VRFs and route-target policy to create separate overlay forwarding contexts. EVPN multihoming adds resilient Ethernet segment connectivity through ESIs, Ethernet Auto-Discovery, DF election and coordinated forwarding procedures.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 07

EVPN Multi-Tenant Fabric Design Lab

Explore EVPN tenant segmentation, VRF routing, route-target import policies and all-active Ethernet segment multihoming. Inject failures, investigate designated-forwarder election and validate tenant isolation and forwarding resilience.

EVPN Multi-Tenancy and Multihoming Laboratory

Animated multi-tenant EVPN fabric architecture

EVPN TENANT VRFs / ROUTE TARGETS / ESI MULTIHOMING EVPN Control Plane MP-BGP / Route Reflector Leaf 1 VTEP / PE 1 Leaf 2 VTEP / PE 2 Leaf 3 Tenant VRFs Dual-Homed Server Shared Ethernet Segment Tenant VRFs BLUE / RED
Multi-tenant fabric ready.
Tenant isolation —
Active PE attachments —
Designated forwarder —
Validation result —

EVPN tenant and Ethernet segment state

AttributeValue
Run fabric validation.

EVPN engineering console

> EVPN MULTI-TENANT FABRIC LAB > READY
Configure tenant and multihoming conditions.
Engineering context: Route targets govern EVPN route import; they are not a complete security boundary by themselves. Tenant isolation also depends on VRF forwarding, route leaking and security policy. In EVPN all-active multihoming, multiple PEs can forward eligible known-unicast traffic, while designated- forwarder procedures govern forwarding of BUM traffic toward an Ethernet segment. Real DF election uses negotiated algorithms and Ethernet-tag-specific procedures; this lab uses a simplified preferred-DF model.
SECTION 08 · EVPN VXLAN SCALABILITY AND HIGH AVAILABILITY

Scaling and Protecting the EVPN VXLAN Fabric

EVPN VXLAN fabrics are designed to support distributed workloads, multiple tenant networks and resilient forwarding across a routed data centre infrastructure.

Scalability and high availability depend on the interaction between physical topology, ECMP forwarding, routing convergence, EVPN control-plane capacity, hardware forwarding resources and failure-domain design.

8.1 — Fabric Scalability Fundamentals

A leaf-spine fabric can scale by increasing the number of leaf switches, spine switches or available interface capacity, subject to platform and topology constraints.

However, physical port capacity is only one dimension of scalability. EVPN route scale, MAC table capacity, VNI support, VRF resources, tunnel endpoints and forwarding hardware limits must also be evaluated.

SCALE DIMENSION 01
Physical Fabric

Leaf count, spine count, uplink bandwidth, port density and oversubscription.

SCALE DIMENSION 02
Control Plane

BGP EVPN sessions, route advertisements, route reflection and convergence.

SCALE DIMENSION 03
Forwarding Hardware

MAC entries, IP routes, VNIs, VRFs, tunnel endpoints and ECMP groups.

8.2 — Interactive EVPN Fabric Resilience Explorer

Select a fabric condition to visualize normal ECMP forwarding, spine failure, leaf uplink failure, route-reflector failure and capacity pressure.

EVPN Fabric Resilience and Capacity Explorer

FABRIC ENGINEERING
ROUTED LEAF-SPINE FABRIC SPINE 1 IP Transit SPINE 2 IP Transit VTEP LEAF LAYER LEAF 1 10.0.0.11 LEAF 2 10.0.0.12 LEAF 3 10.0.0.13 LEAF 4 10.0.0.14 ECMP · CONVERGENCE · REDUNDANCY · CAPACITY
Normal Operation — ECMP Forwarding

Under normal conditions, eligible ECMP paths can carry traffic between leaf switches through multiple spine devices.

8.3 — ECMP and Fabric Capacity

ECMP enables multiple equal-cost routed paths to participate in forwarding.

Increasing the number of spine switches can increase available aggregate fabric bandwidth when sufficient uplinks and compatible forwarding resources exist.

However, ECMP does not guarantee perfectly equal traffic distribution. Flow sizes, hashing behaviour and elephant flows can create uneven utilization.

ILLUSTRATIVE FABRIC CAPACITY Leaf 1 uplinks: Spine 1: 100 Gbps Spine 2: 100 Gbps Aggregate physical uplink capacity: 200 Gbps Actual usable throughput depends on: ECMP hashing Traffic direction Flow distribution Oversubscription Hardware limits Link availability This is not a guaranteed application throughput figure.

8.4 — Route Reflector Scalability

Route reflectors reduce the need for a full mesh of iBGP EVPN sessions between leaf VTEPs.

For large fabrics, redundant route reflectors are commonly deployed to improve control-plane availability.

Route reflector placement and redundancy should account for control-plane failure domains, CPU and memory resources, route processing capacity and convergence.

CONTROL-PLANE SCALE
Route Reflection

Reduces EVPN session-mesh requirements and centralizes route distribution functions.

CONTROL-PLANE RESILIENCE
Redundant Reflectors

Multiple RRs can reduce dependence on a single control-plane distribution point.

8.5 — MAC, IP and EVPN Route Scale

EVPN fabrics may distribute large numbers of MAC/IP advertisements, Ethernet segment routes, IMET routes and IP prefixes.

The route scale must be evaluated against control-plane memory, CPU, routing information bases and hardware forwarding tables.

Resource Scaling concern Engineering action
MAC table Large endpoint populations. Validate hardware MAC scale and aging behaviour.
IP routing table Tenant prefixes and host routes. Check VRF route capacity and FIB resources.
EVPN routes Control-plane processing and memory. Measure route count, churn and RR capacity.
VNI / VRF count Overlay segmentation limits. Verify supported platform scale.
ECMP groups Next-hop and forwarding resources. Check ECMP width and hardware allocation.

8.6 — Failure Domains and Redundancy

Resilient EVPN VXLAN architecture should avoid unnecessary single points of failure.

Important failure domains include physical links, spine devices, leaf devices, route reflectors, power domains, management systems and host attachments.

FAILURE DOMAIN 01
Underlay

Routed uplinks, ECMP paths, physical spines and routing adjacency health.

FAILURE DOMAIN 02
Control Plane

EVPN sessions, route reflectors, route import and next-hop resolution.

FAILURE DOMAIN 03
Workload Access

Leaf attachment, multihoming, gateway availability and local forwarding state.

8.7 — Convergence Engineering

Fabric convergence is the process of detecting a failure, updating routing information and installing usable forwarding paths.

Convergence time depends on failure detection, routing protocol processing, EVPN signalling and forwarding-plane updates.

FAILURE CONVERGENCE SEQUENCE 1. Physical or logical failure 2. Failure detection 3. Underlay routing update 4. Next-hop recalculation 5. Forwarding-plane update 6. Overlay reachability validation 7. Traffic resumes on usable path Additional EVPN updates may be required depending on the failure and service model.

8.8 — High-Availability Design Principles

A high-availability fabric should maintain usable alternate paths and minimize the blast radius of individual component failures.

DESIGN PRINCIPLE 01
Path Redundancy

Connect leaves to multiple spines and validate ECMP behaviour.

DESIGN PRINCIPLE 02
Control-Plane Redundancy

Use resilient EVPN route distribution and verify recovery behaviour.

DESIGN PRINCIPLE 03
Capacity Headroom

Plan for degraded operation when a link or device is unavailable.

DESIGN PRINCIPLE 04
Failure Testing

Validate actual traffic impact during controlled failure scenarios.

8.9 — Production Scalability and HA Troubleshooting

Symptom Likely domain Investigation
Traffic loss after spine failure Underlay convergence or insufficient alternate capacity. Inspect ECMP routes, forwarding updates and remaining uplink utilization.
EVPN route updates delayed Control-plane capacity. Check RR CPU, memory, route processing and session health.
Endpoint learning failures at scale MAC or forwarding resource exhaustion. Inspect hardware table utilization and platform limits.
Uneven spine utilization ECMP hashing and flow distribution. Analyze elephant flows, hashing and available paths.
Unexpected outage during maintenance Failure-domain or redundancy design issue. Validate topology, control-plane redundancy and maintenance procedures.

Engineering takeaway: EVPN VXLAN scalability is not determined by physical port capacity alone. Successful designs must account for control-plane scale, forwarding hardware, ECMP behaviour, redundancy, convergence and capacity during degraded operation.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 08

EVPN Fabric Failure and Convergence Simulator

Investigate leaf-spine failure domains, ECMP resilience, BFD-assisted detection, EVPN route withdrawal and packet loss during convergence. Compare network state before and after a fault and evaluate whether the fabric maintains VTEP-to-VTEP connectivity.

EVPN High Availability and Convergence Laboratory

Animated EVPN high-availability fabric

ECMP FABRIC / FAILURE DOMAINS / CONVERGENCE Spine 1 ECMP Path A Spine 2 ECMP Path B Leaf 1 Source VTEP Leaf 2 Destination VTEP Leaf 3 Additional VTEP Source Leaf 1 → Spine ECMP → Destination Leaf 2
Fabric operational.
Available physical paths —
Estimated convergence —
Estimated lost packets —
Post-failure state —

Network state comparison

Fabric attributeBeforeAfter
Run failure simulation.

Failure and convergence timeline

T+0 ms Fabric initialized and ready for fault injection.

EVPN incident console

> EVPN FAILURE AND CONVERGENCE > READY
Select a failure and run the simulation.
Simulation assumptions: Convergence times and packet-loss estimates are illustrative engineering parameters, not measurements or vendor performance guarantees. BFD affects failure detection but does not itself calculate replacement routes. Physical ECMP path availability, BGP EVPN route state and VTEP health are modeled separately. A fabric can retain underlay connectivity while losing overlay reachability.
SECTION 09 · PRODUCTION EVPN VXLAN TROUBLESHOOTING

Investigating Real-World EVPN VXLAN Failures

EVPN VXLAN troubleshooting requires engineers to investigate multiple interacting layers: physical connectivity, underlay routing, BGP EVPN control-plane advertisements, overlay forwarding state and the encapsulated data plane.

A production incident may appear as a simple connectivity failure, but its root cause could involve missing EVPN routes, incorrect route-target imports, VNI mismatches, unresolved VTEP next hops, MTU limitations or stale forwarding entries.

The most reliable approach is to follow the packet, verify the control-plane evidence and isolate the exact stage where expected forwarding behaviour breaks down.

9.1 — The EVPN VXLAN Troubleshooting Stack

Begin by identifying which part of the architecture is responsible for the observed failure.

LAYER 01
Physical and Underlay

Interfaces, routing adjacencies, ECMP, loopback reachability and transport MTU.

LAYER 02
BGP EVPN Control Plane

Session state, route advertisements, route reflection, route targets and next-hop resolution.

LAYER 03
VXLAN Data Plane

MAC forwarding, VNI mapping, encapsulation, decapsulation and packet delivery.

Operational principle: A BGP EVPN route being present does not guarantee that traffic can be forwarded successfully. The route must be eligible for import, have usable next-hop resolution and result in correct forwarding-plane state.

9.2 — Interactive EVPN VXLAN Incident Investigation Explorer

Select a production incident to inspect the affected architecture, simulated diagnostic output and recommended investigation path.

EVPN VXLAN Incident Investigation Explorer

SIMULATED INCIDENT EVIDENCE
EVPN VXLAN PRODUCTION FABRIC SPINE 1 IP Underlay SPINE 2 IP Underlay LEAF 1 VTEP 10.0.0.11 BGP EVPN / VXLAN LEAF 2 VTEP 10.0.0.12 BGP EVPN / VXLAN HOST A 192.168.10.11 HOST B 192.168.10.22 UNDERLAY · BGP EVPN · VNI · VTEP · FORWARDING
Incident 1 — Underlay Reachability Failure

The ingress VTEP cannot reach the remote VTEP loopback. VXLAN forwarding fails even if previously learned EVPN routes remain visible in the control plane.

Recommended Investigation

9.3 — Underlay Reachability Investigation

VXLAN forwarding depends on IP connectivity between VTEP loopback addresses.

If the underlay cannot reach the destination VTEP, the encapsulated packet cannot be delivered correctly.

UNDERLAY TROUBLESHOOTING Verify: 1. Physical interface status 2. Routing adjacency state 3. VTEP loopback advertisement 4. Route to remote VTEP 5. ECMP next-hop availability 6. Underlay MTU 7. ACL / filtering policy Illustrative checks: show ip route 10.0.0.12 show ip bgp summary ping 10.0.0.12 source 10.0.0.11 Command syntax varies by network platform.

9.4 — BGP EVPN Control-Plane Investigation

BGP EVPN distributes overlay reachability through the L2VPN EVPN address family.

Engineers should verify that the required sessions are established, expected routes are advertised and imported, and the relevant next hops are resolvable.

CONTROL-PLANE CHECK 01
Session State

Confirm BGP EVPN address-family activation and established neighbour sessions.

CONTROL-PLANE CHECK 02
Route Visibility

Verify that expected Type 2, Type 3 or Type 5 routes are received.

CONTROL-PLANE CHECK 03
Route Import

Inspect route targets, import policy and tenant association.

CONTROL-PLANE CHECK 04
Next-Hop Resolution

Verify that remote VTEP reachability is usable for forwarding.

9.5 — Missing MAC/IP Advertisement

EVPN Type 2 routes advertise remote endpoint MAC reachability and optionally associated IP information.

If the route is missing, a remote VTEP may lack the information required for known-unicast forwarding.

TYPE 2 INVESTIGATION Expected endpoint: MAC 00:aa:bb:cc:dd:ee IP 192.168.10.22 Expected remote VTEP: 10.0.0.12 Investigate: 1. Local endpoint learning 2. Type 2 origination 3. EVPN route propagation 4. Route-target import 5. MAC mobility state 6. Remote MAC installation 7. Data-plane forwarding entry Do not assume a received route is installed in hardware.

9.6 — VNI and Route-Target Mismatches

VXLAN forwarding requires consistent service association between the participating VTEPs.

An incorrect VNI mapping can cause traffic to be delivered into the wrong logical context or dropped during validation.

Route-target mismatches can prevent the intended EVPN routes from being imported.

Mismatch Typical symptom Validation
L2VNI mismatch Remote Layer 2 forwarding fails. Compare VLAN-to-VNI mapping and service configuration.
L3VNI mismatch Routed overlay forwarding may fail. Verify VRF-to-L3VNI association.
Route-target mismatch Expected EVPN routes are not imported. Compare route RT attributes and local import policy.
Incorrect VRF association Routes or traffic appear in the wrong tenant context. Inspect VRF membership and routing policy.

9.7 — VXLAN Packet Capture Analysis

Packet captures can help determine whether traffic is encapsulated, transported and decapsulated as expected.

Underlay captures should be examined for outer VTEP addresses, UDP port 4789, VNI values and packet size.

ILLUSTRATIVE VXLAN CAPTURE Outer IPv4: Source: 10.0.0.11 Destination: 10.0.0.12 UDP: Destination Port: 4789 VXLAN: VNI: 10010 Inner Ethernet: Source MAC: 00:11:22:33:44:55 Destination MAC: 00:aa:bb:cc:dd:ee Inner IP: 192.168.10.11 -> 192.168.10.22 Useful Wireshark filter: udp.port == 4789 Capture visibility depends on platform capabilities and capture location.

9.8 — MTU Failures and Silent Packet Drops

VXLAN encapsulation increases packet size, so underlay MTU planning must account for the additional transport headers.

Some MTU-related problems only become visible with larger application packets, making small ping tests misleading.

SYMPTOM
Small Packets Work

Small probes succeed while larger application transfers fail or stall.

INVESTIGATION
End-to-End MTU

Check physical interface MTU, tunnel overhead, packet-size tests and relevant drop counters.

VXLAN MTU ENGINEERING Illustrative IPv4 overhead: Outer Ethernet: 14 bytes Outer IPv4: 20 bytes UDP: 8 bytes VXLAN: 8 bytes ------------------------- Total: 50 bytes This compares an inner Ethernet frame's payload with the outer link transmission size, excluding FCS and optional tags. Actual MTU requirements depend on the packet accounting model, outer IP version and additional headers.

9.9 — Production Incident Investigation Workflow

A repeatable investigation process helps reduce time spent on unrelated symptoms and avoids changing configuration before the failure domain is understood.

EVPN VXLAN INCIDENT WORKFLOW STEP 1 Define affected hosts, tenants and traffic flows. STEP 2 Verify physical links and underlay routing. STEP 3 Validate VTEP loopback reachability. STEP 4 Check BGP EVPN sessions and expected route types. STEP 5 Verify route targets, VNIs and VRF associations. STEP 6 Inspect MAC/IP and next-hop forwarding state. STEP 7 Validate VXLAN packet encapsulation and transport. STEP 8 Check MTU, ACLs, counters and drops. STEP 9 Confirm recovery with end-to-end traffic tests. STEP 10 Record root cause, corrective action and prevention measures.

9.10 — EVPN VXLAN Fault Isolation Matrix

Observed symptom Primary investigation Relevant evidence
Remote VTEP unreachable Underlay routing Loopback route, adjacency and ECMP state
EVPN routes missing BGP control plane Session state, route reflection and advertisements
Remote MAC missing EVPN Type 2 MAC/IP route, import policy and FDB installation
BUM traffic missing EVPN Type 3 IMET membership and replication list
Inter-subnet routing fails IRB and VRF Gateway state, L3VNI, routing table and EVPN reachability
Multihoming blackhole EVPN Type 1/4 ESI, Ethernet AD, DF election and withdrawal state
Large packets fail Transport MTU Packet-size tests and interface drop counters
Unexpected tenant reachability Tenant isolation RT policy, VRF assignment and route leaking

Final engineering takeaway: EVPN VXLAN troubleshooting is most effective when engineers investigate the fabric systematically: underlay first, control plane second, forwarding state third, and packet transport last. Successful troubleshooting requires understanding not only whether a route exists, but whether it is correctly imported, resolved, installed and used to forward real traffic.

NETWORK INSIGHT / ENGINEERING GUIDE 16 / LAB 09

EVPN VXLAN Incident Investigation Simulator

Investigate production-style EVPN VXLAN failures. Run diagnostic commands, inspect BGP EVPN routes, validate VTEP reachability, identify missing MAC entries, detect route-target and VNI mismatches, and apply remediation to restore forwarding.

Advanced EVPN Production Troubleshooting Laboratory

EVPN VXLAN production fabric

EVPN VXLAN PRODUCTION INCIDENT INVESTIGATION Spine Fabric IP Underlay Leaf 1 / Source VTEP 10.0.0.11 Leaf 2 / Destination VTEP 10.0.0.12
Incident laboratory ready.
Incident state —
Diagnostics run 0
Root cause —
Engineering score 0

Incident state comparison

Network attributeIncidentAfter remediation
Load an incident.

Investigation timeline

READY Load an incident to begin.

Network engineering CLI

> EVPN VXLAN INCIDENT LAB > READY
Load an incident and begin troubleshooting.
Operational guidance: Troubleshoot EVPN VXLAN in layers: physical interfaces and underlay routing, VTEP loopback reachability, MP-BGP EVPN sessions, route advertisement and import, VNI/VRF mapping, MAC/IP forwarding state, MTU and Ethernet segment behavior. Command syntax and outputs are illustrative and vary by network operating system. A successful BGP session does not guarantee correct data-plane forwarding.
☕
Support Network Insight
Help support the development of interactive networking labs, BGP simulations, and educational content.
Support the Project
Matt Conran
Tags: No tags

Comments are closed.