Overlay Virtual Networking Vendors

Network Overlays


overlay tunnel


Network Overlays

Network overlays have emerged as a powerful solution to address the challenges posed by the increasing complexity of modern networks. This blog post will explore network overlays, their benefits, and how they improve connectivity and scalability in today’s digital landscape.

Network overlays are virtual networks that run on physical networks, providing an additional abstraction layer. They allow organizations to create logical networks independent of the underlying physical infrastructure. This decoupling enables flexibility, scalability, and simplified management of complex network architectures.

Network overlays refer to virtualizing network services and infrastructure over existing physical networks. By decoupling the network control plane from the underlying hardware, network overlays provide a layer of abstraction that simplifies network management while offering enhanced flexibility and scalability. This approach allows organizations to create virtual networks tailored to their specific needs without the constraints imposed by physical infrastructure limitations.

Highlights: Network Overlays

  • Creating an overlay tunnel

A network overlay is an architecture that creates a virtualized network on top of an existing physical network. It allows multiple virtual networks to run independently and securely on the same physical infrastructure. Network overlays are a great way to create a more secure and flexible network environment without investing in new infrastructure.

Network overlays can be used for various applications, such as creating virtual LANs (VLANs), virtual private networks (VPNs), and multicast networks. For example, we have DMVPN (Dynamic Multipoint VPN), with several DMVPN phases providing a secure network technology that allows for multiple sites’ efficient and secure connection.

In addition, they can be used to segment traffic and provide secure communication between two or more networks. As a result, network overlays allow for more efficient use of resources and provide better performance, scalability, and security.

  • Enhanced Connectivity:

Network overlays improve connectivity by enabling seamless communication between different network domains. By abstracting the underlying physical infrastructure, overlays facilitate the creation of virtual network segments that can span geographical locations, data centers, and cloud environments. This enhanced connectivity promotes better collaboration, data sharing, and application access within and across organizations.


Related: Before you proceed, you may find the following helpful:

  1. Data Center Topologies
  2. SD WAN Overlay
  3. Nexus 1000v
  4. SDN Data Center
  5. Virtual Overlay Network


Overlay Tunnel

Key Network Overlays Discussion Points:

  • Introduction to network overlays and what is involved.

  • Highlighting the details of control plane interaction.

  • Technical details on the encapsulation overhead.

  • Scenario: Security in the tunnel overlay.

  • A final note on STP and layer 2 attacks.


  • A key point: Video on overlay tunnel and VXLAN phases.

In this video, we will discuss the VLXAN phases. VXLAN went through several steps for ways to get the remote VTEP IP information. Initially, it started with a flood and learning the process and finally moved to use a proper control plane – EVPN.

EVPN is a pretty good control plan, as previous methods rely on data plane Flood and Learn behavior, hindering the scalability of VXLAN domains. It also has several security features that can secure the tunnel overlay.



Back to Basics With Network Overlays

Supporting distributed application

There has been a significant paradigm shift in data center networks. This evolution has driven network overlays known as tunnel overlay, bringing several new requirements to data center designs. Distributed applications are transforming traffic profiles, and there is a rapid rise in intra-DC traffic ( East-West ).

To support this type of scale, we, designers, face several challenges. First, there is no choice but to implement network virtualization with the overlay tunnel for large cloud deployment.

Suppose a customer requires a logical segment per application, and each application requires load balancing or firewall services between segments. In that case, having an all-physical network using traditional VLANs is impossible. The limitations of 4000 VLANS and the requirement for stretched Layer 2 subnets have pushed designers to virtualize workloads over an underlying network.


Lab guide on network overlay with VXLAN

The following guide has a Layer 2 overlay across a routed Layer 3 core. Spine A and Spine B cores run an OSPF network to each other and the leaf layer. VXLAN is the overlay protocol that maps a bridge domain to a VNI that extends layer 2 across the routed core. Notice in the show command output that the encapsulation is set to VXLAN.


VXLAN overlay
Diagram: VXLAN Overlay

Concepts of network Virtualization

Network virtualization is cutting up a single physical network into multiple virtual networks. Virtualizing a resource allows it to be shared by various users. Numerous virtual networks have jumped up over the decades to satisfy different needs. 

A primary distinction between these different types is their model for providing network connectivity. Networks can provide connectivity via bridging (L2) or routing (L3). Thus, virtual networks can be either virtual L2 networks or virtual L3 networks.

Virtual networks started with the Virtual Local Area Network (VLAN). First, the VLAN was invented to lessen the unnecessary chatter in a Layer 2 network by isolating applications from their noisy neighbors. Then VLAN was then pushed into the world of security.

Then, we had the Virtual Routing and Forwarding (VRF). The virtual L3 network was invented along with the L3 Virtual Private Network (L3VPN) to solve the problem of interconnecting geographically disparate networks of an enterprise over a public network. 

Network Overlay

Main Network Overlay Components

Network Overlay

  • Network overlays are virtual networks that run on physical networks.

  • Network overlays improve connectivity by enabling seamless communication between different network domains.

  • The limitations of 4000 VLANS and the requirement for stretched Layer 2 subnets have pushed designers to virtualize workloads over an underlying network.

  • Virtual networks can be either virtual L2 networks or virtual L3 networks.

Benefits of Network Overlays:

1. Simplified Network Management: With network overlays, organizations can manage their networks centrally, using software-defined networking (SDN) controllers. This centralized approach eliminates the need for manual configuration and reduces the complexity associated with traditional network management.

2. Enhanced Scalability: Network overlays enable businesses to scale their networks easily by provisioning virtual networks on demand. This flexibility allows rapid deployment of new services and applications without physical network reconfiguration.

3. Improved Security: Network overlays provide an additional layer of security by encapsulating traffic within virtual tunnels. This isolation helps prevent unauthorized access and reduces the risk of potential security breaches, especially in multi-tenant environments.

4. Interoperability: Network overlays can be deployed across heterogeneous environments, enabling seamless connectivity between different network types, such as private and public clouds. This interoperability makes it possible to extend the network across multiple locations and integrate various cloud services effortlessly.

  • Scalability and Elasticity

One of the critical advantages of network overlays is their ability to scale and adapt to changing network requirements. Overlays can dynamically allocate resources based on demand, allowing network administrators to deploy new services or expand existing ones rapidly. This elasticity enables organizations to meet the evolving needs of their users and applications without the constraints imposed by the underlying physical infrastructure.

  • Simplified Network Management

Network overlays simplify network management by providing a centralized control plane. This control plane abstracts the complexity of the underlying physical infrastructure, allowing administrators to configure and manage the virtual networks through a single interface. This simplification reduces operational overhead, minimizes human errors, and enhances network security.


So one of the first notable differences between VXLAN vs VLAN was increased scalability, as the VXLAN ID is 24 bits, enabling you to create up to 16 million isolated networks. This overcomes the limitation of VLANs having the 12-bit VLAN ID, which enables only a maximum of 4094 isolated networks.

Tunnel Overlay
Multiple segments per application and the need for a tunnel overlay,


What are the drawbacks of network overlays, and how does it affect network stability?


Use Cases of Network Overlays:

1. Data Center Networking: Network overlays are commonly used in data center environments to simplify network management, enhance scalability, and improve workload mobility. By abstracting the physical network infrastructure, data center operators can create isolated virtual networks for different applications or tenants, ensuring optimal resource utilization and agility.

2. Cloud Networking: Network overlays enable connectivity and security in cloud environments. Whether public, private, or hybrid clouds, network overlays allow organizations to extend their networks seamlessly, ensuring consistent connectivity and security policies across diverse cloud environments.

3. Multi-site Connectivity: For organizations with multiple locations, network overlays provide a cost-effective solution for connecting geographically dispersed sites. By leveraging virtual networks, businesses can establish secure and reliable communication channels between sites, regardless of the underlying physical infrastructure.

Overlay Technologies:

1.Virtual Private Networks (VPNs):

Virtual Private Networks create secure, encrypted connections over public networks, enabling remote access and secure communication between geographically dispersed locations. VPNs are commonly used to provide secure connectivity for remote workers or to establish site-to-site connections between different branches of an organization.

2.Software-Defined Networking (SDN):

Software-defined networking is a network architecture that separates the control plane from the data plane, enabling centralized network management and programmability. SDN overlays can leverage network virtualization techniques to create logical networks independent of the underlying physical infrastructure.

3.Network Function Virtualization (NFV):

Network Function Virtualization abstracts network services, such as firewalls, load balancers, and routers, from dedicated hardware appliances and runs them as virtual instances. NFV overlays allow organizations to deploy dynamically and scale network services, reducing costs and improving operational efficiency.


Control Plane Interaction

Tunneled network overlays

Virtualization adds a level of complexity to the network. Consider the example of a standard tunnel. We are essentially virtualizing workloads over an underlying network. From a control plane perspective, there must be more than one control plane.

This results in two views of the network’s forwarding and reachability information – a view from the tunnel endpoints and a view from the underlying network. The control plane may be static or dynamic and provides reachability through the virtual topology on top of the control plane that provides reachability to the tunnel endpoints.

overlay tunnel
The overlay tunnel and potential consequences.


Router A has two paths to reach Already, we have the complexity of influencing and managing what traffic should and shouldn’t go down the tunnel. Modifying metrics for specific destinations will influence path selection, but this comes with additional configuration complexity and policies’ manual management.

The incorrect interaction configuration between two control planes may cause a routing loop or suboptimal routing through the tunnel interfaces. The “routers in the middle” and the “routers at tunnel edges” have different views of the network – increasing network complexity.


  • A key point: Not an independent topology

These two control planes may seem to act independently, but it is not an independent topology. The control plane of the virtual topology relies heavily on the control plane of the underlying network. These control planes should not be allowed to interplay freely, as both can react differently to inevitable failures. The timing of the convergence process and how quickly each control plane reacts may be the same or different.

The underlying network could converge faster or slower than the overlaying control plane, affecting application performance. Design best practice is to design the network overlays control plane so that it detects and reacts to network failures faster than the underlying control plane or have the underlying control plane detect and respond faster than the network overlays control plane.

Encapsulation overhead

Every VXLAN packet originating from the end host and sent toward the IP core will be stamped with a VXLAN header. This leads to an additional 50 bytes per packet from the source to the destination server. If the core cannot accommodate the greater MTU size or the Path MTU is broken, it may have to fragment the packet into smaller pieces. Also, the VXLAN header must be encapsulated and de-encapsulated on the virtual switch, which takes up computing cycles. Both of these are problematic for network performance.

vxlan overhead
VXLAN overhead.

Security in a tunnel overlay

There are many security flaws with tunnels and network overlays. The most notable is that they hide path information. A tunnel can pass one route on one day and take another path on a different day. The change of path may be unknown to the network administrator. Traditional routing is hop-by-hop; every router decides where the traffic should be routed.

However, the independent hop-by-hop decisions are not signaled and known by the tunnel endpoints. As a result, an attacker can direct the tunnel traffic via an unintended path where the rerouted traffic can be monitored and snooped.

VXLAN security

Tunneled traffic hides from any policies or security checkpoints. Many firewalls have HTTP port 80 open to support web browsing. This can allow an attacker to tunnel traffic in an HTTP envelope, bypassing all the security checks. There are also several security implications if you are tunneling with GRE.

First, GRE performs no encryption on any part of the data journey or authentication. The optional 32-bit tunnel key for identifying individual traffic flows can easily be brute-forced due to the restriction of 2×32 number combinations.

Finally, it has a weak implementation of the sequence number used to provide a method of in-order delivery. These shortcomings have opened up to several MTU-based and GRE packet injection attacks.

 STP and Layer 2 attacks

VXLAN extends layer 2 domains across layer 3 boundaries, resulting in more extensive layer 2 flat networks. Regarding intrusion, the attack zones become much more significant as we connect up to two remote disjointed endpoints. This increases the attack zones over traditional VLANs where the Layer 2 broadcast domain was much smaller.

You are open to various STP attacks if you run STP over VXLAN. Tools such as BSD brconfig and Linux bridge-utilis allow you to generate STP frames into a Layer 2 network and can be used to insert a rogue root bridge to modify the traffic path.

 Tunnel overlay with VXLAN inbuilt security?

The VXLAN standard has no built-in security, so if your core is not secure and becomes compromised, so will all your VXLAN tunneled traffic. Schemes such as 802.1x should be deployed for the admission control of VTEP ( tunnel endpoints ). 802.1x at the edges provides defense so that rogue endpoints may not inject traffic into the VXLAN cloud. The VXLAN payload can also be encrypted with IPsec.


Summary: Understanding Network Overlays

At its core, a network overlay is a virtual network created using software-defined networking (SDN) technologies. It enables the creation of logical network segments independent of the physical infrastructure. By decoupling the network’s control plane from its data plane, overlays provide network architectures flexibility, scalability, and agility.

Benefits of Network Overlays

Enhanced Security and Isolation

Network overlays provide strong isolation between virtual networks, ensuring that traffic remains separate and secure. This isolation helps protect sensitive data and prevents unauthorized access, making overlays an ideal solution for multi-tenant environments.

Simplified Network Management

With network overlays, administrators can manage and control the network centrally, regardless of the underlying physical infrastructure. This centralized management simplifies network provisioning, configuration, and troubleshooting, improving operational efficiency.

Overlay Technologies

Virtual Extensible LAN (VXLAN)

VXLAN is a widely adopted overlay technology that extends Layer 2 networks over an existing Layer 3 infrastructure. It uses encapsulation techniques to provide scalability and flexibility, allowing for the seamless expansion of network segments.

Generic Routing Encapsulation (GRE)

GRE is another popular overlay protocol that enables the creation of private point-to-point or multipoint tunnels over an IP network. It provides a simple and reliable way to connect geographically dispersed networks and facilitates the transit of diverse protocols.

Use Cases for Network Overlays

Data Center Virtualization

Network overlays play a crucial role in data center virtualization, allowing for the creating of virtual networks that can span multiple physical servers. This enables efficient resource utilization, workload mobility, and simplified network management.

Hybrid Cloud Connectivity

Organizations can establish secure and seamless connections between on-premises infrastructure and public cloud environments by leveraging network overlays. This enables hybrid cloud architectures, providing the flexibility to leverage the benefits of both worlds.

Conclusion: In conclusion, network overlays have emerged as a powerful tool in modern networking, enabling flexibility, security, and simplified management. With their ability to abstract away the complexities of physical infrastructure, overlays pave the way for innovative network architectures. As technology continues to evolve, network overlays will undoubtedly play a vital role in shaping the future of networking.


overlay tunnel


Matt Conran
Latest posts by Matt Conran (see all)

Comments are closed.