Overlay networks are computer networks that are layered on top of other networks (logical instead of physical). They differ from the traditional OSI layered network model and almost always assume that the underlay network is an IP network. These technologies include VXLAN, BGP VPNs, Layer 2 and Layer 3, and IP over IP, such as GRE or IPsec tunnels. Overlay networks, such as SD-WAN, use IP over IP technologies.

The overlay network (SDN overlay) allows multiple network layers to be run on top of each other, adding new applications and improving security. Multiple secure overlays can be created using software over existing networking hardware infrastructure by making virtual connections between two endpoints. Endpoints can be physical locations, such as network ports, or logical locations, such as software addresses, in the cloud.

Software tags, labels, and/or encryption create a virtual tunnel between two network endpoints. If encryption is used, end users must be authenticated to use the connection. Similar to a phone system, the technology can be considered endpoints with identification tags. An identification tag or number can be used to locate a device in a network, creating virtual connections.

Networking approach based on overlays

Different overlay networking approaches are often debated in the SDN community. Some software-only solutions may not be able to integrate at the chip level, depending on the technology. The layering of software and processing in overlay networking has been criticized for creating performance overhead. Network overlays are controlled by SDN controllers using the OpenFlow protocol, which requires specific software code or “agents” to be installed.

Change in Traffic Patterns

Despite being in remote geographic locations, a host of physical servers and I/O devices can host multiple virtual servers that share the same logical network, thanks to the paradigm shift toward cloud computing. In contrast to the traditional north-south direction of data traffic within data centers, virtualization has facilitated more significant east-west data traffic. Communication between servers and applications within a data center is known as east-west traffic.

In corporate networks or on the Internet, much of the data required by the end user involves more complex data that requires preprocessing. Using a web server (via an app server) to access a database as an example of east-west traffic, we can demonstrate the need for preprocessing.

The birth of network overlays

Network virtualization overlays have become the de facto solution for addressing the problems just described in relation to data center expansion. Overlays allow existing network technologies to be abstracted, extending the capabilities of classic networks.

Networking has been using overlays for quite some time. Overlays were developed because of the disadvantages of conventional networks. An overlay is a tunnel that runs on top of a physical network infrastructure, as its name implies.

Following MPLS- and GRE-based encapsulations in the 1990s, other tunneling technologies, such as IPsec,8 6in4,9, and L2TPv3,10, also gained popularity. For example, 6in4 tunnels were used to carry payloads over a transport network that could not support the payload type. These tunnels were utilized for security purposes, simplifying routing lookups, or carrying payloads over unsupported transport networks.

Network Overlays and Virtual Networks

Network overlays have emerged as a powerful solution to address the challenges posed by the increasing complexity of modern networks. This blog post will explore network overlays, their benefits, and how they improve connectivity and scalability in today’s digital landscape.

Network overlays are virtual networks that run on physical networks, providing an additional abstraction layer. They allow organizations to create logical networks independent of the underlying physical infrastructure. This decoupling enables flexibility, scalability, and simplified management of complex network architectures.

Virtual Network Services

Network overlays refer to virtualizing network services and infrastructure over existing physical networks. By decoupling the network control plane from the underlying hardware, network overlays provide a layer of abstraction that simplifies network management while offering enhanced flexibility and scalability. This approach allows organizations to create virtual networks tailored to their specific needs without the constraints imposed by physical infrastructure limitations.

Creating an overlay tunnel

A network overlay is an architecture that creates a virtualized network on top of an existing physical network. It allows multiple virtual networks to run independently and securely on the same physical infrastructure. Network overlays are a great way to create a more secure and flexible network environment without investing in new infrastructure.

Network overlays can be used for various applications, such as creating virtual LANs (VLANs), virtual private networks (VPNs), and multicast networks. For example, we have DMVPN (Dynamic Multipoint VPN), with several DMVPN phases providing a secure network technology that allows for multiple sites’ efficient and secure connection.

In addition, they can segment traffic and provide secure communication between two or more networks. As a result, network overlays allow for more efficient resource use and provide better performance, scalability, and security.

Enhanced Connectivity:

Network overlays improve connectivity by enabling seamless communication between different network domains. By abstracting the underlying physical infrastructure, overlays facilitate the creation of virtual network segments that can span geographical locations, data centers, and cloud environments. This enhanced connectivity promotes better collaboration, data sharing, and application access within and across organizations.

Related: Before you proceed, you may find the following helpful:

1. Data Center Topologies
2. SD WAN Overlay
3. Nexus 1000v
4. SDN Data Center
5. Virtual Overlay Network
6. SD WAN SASE

Back to Basics With Network Overlays

Supporting distributed application

There has been a significant paradigm shift in data center networks. This evolution has driven network overlays known as tunnel overlay, bringing several new requirements to data center designs. Distributed applications are transforming traffic profiles, and there is a rapid rise in intra-DC traffic ( East-West ).

We designers face several challenges to support this type of scale. First, we must implement network virtualization with the overlay tunnel for large cloud deployment.

Suppose a customer requires a logical segment per application, and each application requires load balancing or firewall services between segments. In that case, having an all-physical network using traditional VLANs is impossible. The limitations of 4000 VLANS and the requirement for stretched Layer 2 subnets have pushed designers to virtualize workloads over an underlying network.

1st Lab guide on network overlay with VXLAN

The following guide has a Layer 2 overlay across a routed Layer 3 core. Spine A and Spine B cores run an OSPF network to each other and the leaf layer. VXLAN is the overlay protocol that maps a bridge domain to a VNI that extends layer 2 across the routed core. Notice in the show command output that the encapsulation is set to VXLAN.

Concepts of network Virtualization

Network virtualization is cutting up a single physical network into multiple virtual networks. Virtualizing a resource allows it to be shared by various users. Numerous virtual networks have jumped up over the decades to satisfy different needs. 

A primary distinction between these different types is their model for providing network connectivity. Networks can provide connectivity via bridging (L2) or routing (L3). Thus, virtual networks can be either virtual L2 networks or virtual L3 networks.

Virtual networks started with the Virtual Local Area Network (VLAN). First, the VLAN was invented to lessen the unnecessary chatter in a Layer 2 network by isolating applications from their noisy neighbors. Then VLAN was then pushed into the world of security.

Then, we had the Virtual Routing and Forwarding (VRF). The virtual L3 network was invented along with the L3 Virtual Private Network (L3VPN) to solve the problem of interconnecting geographically disparate networks of an enterprise over a public network. 

Benefits of Network Overlays:

1. Simplified Network Management: With network overlays, organizations can manage their networks centrally, using software-defined networking (SDN) controllers. This centralized approach eliminates the need for manual configuration and reduces the complexity associated with traditional network management.

2. Enhanced Scalability: Network overlays enable businesses to scale their networks easily by provisioning virtual networks on demand. This flexibility allows rapid deployment of new services and applications without physical network reconfiguration.

3. Improved Security: Network overlays provide an additional layer of security by encapsulating traffic within virtual tunnels. This isolation helps prevent unauthorized access and reduces the risk of potential security breaches, especially in multi-tenant environments.

4. Interoperability: Network overlays can be deployed across heterogeneous environments, enabling seamless connectivity between different network types, such as private and public clouds. This interoperability makes it possible to extend the network across multiple locations and integrate various cloud services effortlessly.

• Scalability and Elasticity

One of the critical advantages of network overlays is their ability to scale and adapt to changing network requirements. Overlays can dynamically allocate resources based on demand, allowing network administrators to deploy new services or expand existing ones rapidly. This elasticity enables organizations to meet the evolving needs of their users and applications without the constraints imposed by the underlying physical infrastructure.

• Simplified Network Management

Network overlays simplify network management by providing a centralized control plane. This control plane abstracts the complexity of the underlying physical infrastructure, allowing administrators to configure and manage the virtual networks through a single interface. This simplification reduces operational overhead, minimizes human errors, and enhances network security.

• VXLAN vs VLAN

One of the first notable differences between VXLAN and VLAN was increased scalability. The VXLAN ID is 24 bits, enabling you to create up to 16 million isolated networks. This overcomes the limitation of VLANs having the 12-bit VLAN ID, which enables only a maximum of 4094 isolated networks.

What are the drawbacks of network overlays, and how does it affect network stability?

Use Cases of Network Overlays:

1. Data Center Networking: Network overlays are commonly used in data center environments to simplify network management, enhance scalability, and improve workload mobility. By abstracting the physical network infrastructure, data center operators can create isolated virtual networks for different applications or tenants, ensuring optimal resource utilization and agility.

2. Cloud Networking: Network overlays enable connectivity and security in cloud environments. Whether public, private, or hybrid clouds, network overlays allow organizations to extend their networks seamlessly, ensuring consistent connectivity and security policies across diverse cloud environments.

3. Multi-site Connectivity: For organizations with multiple locations, network overlays provide a cost-effective solution for connecting geographically dispersed sites. By leveraging virtual networks, businesses can establish secure and reliable communication channels between sites, regardless of the underlying physical infrastructure.

Overlay Technologies:

1. Virtual Private Networks (VPNs):

Virtual Private Networks create secure, encrypted connections over public networks, enabling remote access and secure communication between geographically dispersed locations. VPNs are commonly used to provide secure connectivity for remote workers or to establish site-to-site connections between different branches of an organization.

2. Software-Defined Networking (SDN):

Software-defined networking is a network architecture that separates the control plane from the data plane, enabling centralized network management and programmability. SDN overlays can leverage network virtualization techniques to create logical networks independent of the underlying physical infrastructure.

3. Network Function Virtualization (NFV):

Network Function Virtualization abstracts network services, such as firewalls, load balancers, and routers, from dedicated hardware appliances and runs them as virtual instances. NFV overlays allow organizations to deploy dynamically and scale network services, reducing costs and improving operational efficiency.

Control Plane Interaction

Tunneled network overlays

Virtualization adds a level of complexity to the network. Consider the example of a standard tunnel. We are essentially virtualizing workloads over an underlying network. From a control plane perspective, there must be more than one control plane.

This results in two views of the network’s forwarding and reachability information—a view from the tunnel endpoints and a view from the underlying network. The control plane may be static or dynamic and provides reachability through the virtual topology on top of it, which provides reachability to the tunnel endpoints.

Router A has two paths to reach 192.0.2.0/24. Already, we have the complexity of influencing and managing what traffic should and shouldn’t go down the tunnel. Modifying metrics for specific destinations will influence path selection, but this comes with additional configuration complexity and policies’ manual management.

The incorrect interaction configuration between two control planes may cause a routing loop or suboptimal routing through the tunnel interfaces. The “routers in the middle” and the “routers at tunnel edges” have different views of the network – increasing network complexity.

• A key point: Not an independent topology

These two control planes may seem to act independently, but they are not independent topologies. The control plane of the virtual topology relies heavily on the control plane of the underlying network. These control planes should not be allowed to interplay freely, as both can react differently to inevitable failures. The timing of the convergence process and how quickly each control plane reacts may be the same or different.

The underlying network could converge faster or slower than the overlaying control plane, affecting application performance. Design best practice is to design the network overlays control plane so that it detects and reacts to network failures faster than the underlying control plane or have the underlying control plane detect and respond faster than the network overlays control plane.

Encapsulation overhead

Every VXLAN packet originating from the end host and sent toward the IP core will be stamped with a VXLAN header. This leads to an additional 50 bytes per packet from the source to the destination server. If the core cannot accommodate the greater MTU size or the Path MTU is broken, the packet may have to be fragmented into smaller pieces. Also, the VXLAN header must be encapsulated and de-encapsulated on the virtual switch, which takes up computing cycles. Both of these are problematic for network performance.

Security in a tunnel overlay

There are many security flaws with tunnels and network overlays. The most notable is that they hide path information. A tunnel can pass one route on one day and take another path on a different day. The change of path may be unknown to the network administrator. Traditional routing is hop-by-hop; every router decides where the traffic should be routed.

However, independent hop-by-hop decisions are not signaled and are not known by the tunnel endpoints. As a result, an attacker can direct the tunnel traffic via an unintended path where the rerouted traffic can be monitored and snooped.

Tunneled traffic hides from any policies or security checkpoints. Many firewalls have HTTP port 80 open to support web browsing. This can allow an attacker to tunnel traffic in an HTTP envelope, bypassing all the security checks. There are also several security implications if you are tunneling with GRE.

First, GRE does not perform encryption or authentication on any part of the data journey. The optional 32-bit tunnel key for identifying individual traffic flows can easily be brute-forced due to the restriction of 2×32 number combinations.

Finally, it has a weak implementation of the sequence number used to provide a method of in-order delivery. These shortcomings have opened up to several MTU-based and GRE packet injection attacks.

 STP and Layer 2 attacks

VXLAN extends layer 2 domains across layer 3 boundaries, resulting in more extensive layer 2 flat networks. Regarding intrusion, the attack zones become much more significant as we connect up to two remote disjointed endpoints. This increases the attack zones over traditional VLANs where the Layer 2 broadcast domain was much smaller.

You are open to various STP attacks if you run STP over VXLAN. Tools such as BSD brconfig and Linux bridge-utilis allow you to generate STP frames into a Layer 2 network and can be used to insert a rogue root bridge to modify the traffic path.

 Tunnel overlay with VXLAN inbuilt security?

The VXLAN standard has no built-in security, so if your core is not secure and becomes compromised, so will all your VXLAN tunneled traffic. Schemes such as 802.1x should be deployed for the admission control of VTEP ( tunnel endpoints ). 802.1x at the edges provides defense so that rogue endpoints may not inject traffic into the VXLAN cloud. The VXLAN payload can also be encrypted with IPsec.

Closing Points: Understanding Network Overlays

At its core, a network overlay is a virtual network created using software-defined networking (SDN) technologies. It enables the creation of logical network segments independent of the physical infrastructure. By decoupling the network’s control plane from its data plane, overlays provide flexibility, scalability, and agility for network architectures.

Benefits of Network Overlays

Enhanced Security and Isolation

Network overlays provide strong isolation between virtual networks, ensuring that traffic remains separate and secure. This isolation helps protect sensitive data and prevents unauthorized access, making overlays an ideal solution for multi-tenant environments.

Simplified Network Management

With network overlays, administrators can manage and control the network centrally, regardless of the underlying physical infrastructure. This centralized management simplifies network provisioning, configuration, and troubleshooting, improving operational efficiency.

Overlay Technologies

Virtual Extensible LAN (VXLAN)

VXLAN is a widely adopted overlay technology that extends Layer 2 networks over an existing Layer 3 infrastructure. It uses encapsulation techniques to provide scalability and flexibility, allowing for the seamless expansion of network segments.

Generic Routing Encapsulation (GRE)

GRE is another popular overlay protocol that enables the creation of private point-to-point or multipoint tunnels over an IP network. It provides a simple and reliable way to connect geographically dispersed networks and facilitates the transit of diverse protocols.

Use Cases for Network Overlays

Data Center Virtualization

Network overlays play a crucial role in data center virtualization, allowing the creation of virtual networks that can span multiple physical servers. This enables efficient resource utilization, workload mobility, and simplified network management.

Hybrid Cloud Connectivity

By leveraging network overlays, organizations can establish secure and seamless connections between on-premises infrastructure and public cloud environments. This enables hybrid cloud architectures, providing the flexibility to leverage the benefits of both worlds.

In conclusion, network overlays have emerged as a powerful tool in modern networking, enabling flexibility, security, and simplified management. With their ability to abstract away the complexities of physical infrastructure, overlays pave the way for innovative network architectures. As technology continues to evolve, network overlays will undoubtedly play a vital role in shaping the future of networking.