rsz_overlay_soltuins

Overlay Virtual Networking | Overlay Virtual Networks

Overlay Virtual Networks

In today's interconnected world, networks enable seamless communication and data transfer. Overlay virtual networking has emerged as a revolutionary approach to network connectivity, offering enhanced flexibility, scalability, and security. This blog post aims to delve into the concept of overlay virtual networking, exploring its benefits, use cases, and potential implications for modern network architectures.

Overlay virtual networking is a network virtualization technique that decouples the logical network from the underlying physical infrastructure. It creates a virtual network on top of the existing physical infrastructure, enabling the coexistence of multiple logical networks on the same physical infrastructure. By abstracting the network functions and services from the physical infrastructure, overlay virtual networking provides a flexible and scalable solution for managing complex network environments.

- Scalability and Flexibility: Overlay virtual networks provide the ability to scale network resources on-demand without disrupting the underlying physical infrastructure. This enables organizations to expand their network capabilities swiftly and efficiently, catering to changing business requirements.

- Enhanced Security: Overlay virtual networks offer heightened security by isolating traffic and providing secure communication channels. By segmenting the network into multiple virtual domains, potential threats can be contained, preventing unauthorized access to sensitive data.

- Cloud Computing: Overlay virtual networks are extensively used in cloud computing environments. They allow multiple tenants to have their own isolated virtual networks, ensuring data privacy and security. Additionally, overlay networks enable seamless migration of virtual machines between physical hosts, enhancing resource utilization.

- Software-Defined Networking (SDN): Overlay virtual networks align perfectly with the principles of Software-Defined Networking. By abstracting the logical network from the physical infrastructure, SDN controllers can dynamically manage and provision network resources, optimizing performance and efficiency.

Conclusion: Overlay virtual networks have emerged as a powerful networking solution, providing scalability, flexibility, and enhanced security. Their applications span across various domains, including cloud computing and software-defined networking. As technology continues to evolve, overlay virtual networks are poised to play a vital role in shaping the future of networking.

Highlights: Overlay Virtual Networks

Overlay Network Architecture

Overlay virtual networks are built on the existing physical network infrastructure, creating a logical network layer that operates independently. This architecture allows organizations to leverage the benefits of virtualization without disrupting their underlying network infrastructure.

The virtual network overlay software is at the heart of an overlay virtual network. This software handles the encapsulation and decapsulation of network packets, enabling communication between virtual machines (VMs) or containers across different physical hosts or data centers. It ensures data flows seamlessly within the overlay network, regardless of the underlying network topology.

Tunneling Protocols

Tunneling protocols play a crucial role in overlay virtual networks by facilitating the encapsulation and transportation of network packets over the underlying physical network. Popular tunneling protocols such as VXLAN (Virtual Extensible LAN), NVGRE (Network Virtualization using Generic Routing Encapsulation), and Geneve provide the necessary mechanisms for creating virtual tunnels and encapsulating traffic.

The Network Virtualization Edge (NVE) acts as the endpoint for the overlay virtual network. It connects the physical network infrastructure to the virtual network, ensuring seamless communication between the two. NVEs perform functions like encapsulation, decapsulation, and mapping virtual network identifiers (VNIs) to the appropriate virtual machines or containers.

Example: GRE and the tunneling protocol

GRE, or Generic Routing Encapsulation, is a tunneling protocol widely used in overlay networks. It encapsulates various network layer protocols within IP packets, enabling virtual point-to-point connections over an existing IP network. GRE provides a mechanism to extend private IP addressing schemes over public networks, facilitating secure and efficient communication between remote locations.

GRE without IPsec

 

Overlay Network Control Plane

The control plane in an overlay virtual network manages and maintains the overall network connectivity. It handles tasks such as route distribution, network mapping, and keeping the overlay network’s forwarding tables. Border Gateway Protocol (BGP) and Virtual Extensible LAN Segment Identifier (VXLAN VNI) provide the necessary control plane mechanisms.

Overlays and a Clos Fabric

The underlay of most modern data centers is a 3-stage or 5-stage Clos fabric, with the physical infrastructure and point-to-point Layer 3 interfaces between the spines and leaves. Network virtualization can be created by elevating the endpoints and applications connected to the network into this overlay, thus logically carving out different services on top of it.

Several factors have contributed to the need for network virtualization in such an architecture:

  • Virtualization of hosts and multitenancy have become commonplace in today’s data centers. Building and maintaining a unique physical infrastructure for each tenant is naturally costly.
  • Furthermore, maintaining a state for 100,000 hosts or more may not be feasible in most environments. The network’s core (the fabric’s spines, in this case) can be a pure IP forwarding device without any knowledge of the endpoints.
  • A 4K VLAN range was often a barrier in multitenant environments, especially in cloud computing.
  • Host mobility forces networks to retain constructs such as network gateways to ensure the mobility function does not break across leaf boundaries.
  • Legacy applications, host mobility, and storage clusters drive the need for a flat Layer 2 network over a routed infrastructure.
  • In network virtualization, packets are encapsulated as they enter the network, creating a network overlay. As a result, a physical infrastructure is covered with a logical network. Overlays enable the construction and connection of logically separated and isolated networks atop the same physical network.

Network overlay

In RFC 7348, a Virtual Extensible LAN (VXLAN) is a data plane encapsulation type capable of supporting Layer 2 and Layer 3 payloads. In addition to logically separating broadcast or bridging domains in a network, virtual LANs (VLANs) are limited in their scalability to 4K VLANs. By contrast, VXLAN provides a 24-bit VXLAN Network Identifier (VNI) in the VXLAN header, allowing the network administrator more flexibility to partition the network logically.

VXLAN is, in essence, a stateless tunnel, originating at one endpoint and terminating at another because of its encapsulating trait. The VXLAN Tunnel Endpoints (VTEPs) are the endpoints that encapsulate and decapsulate the VXLAN tunnel. The first thing you need to understand about VXLAN is that these tunnels can originate and terminate on network devices or servers with the help of a virtual switch such as Open vSwitch, with a VXLAN module that is usually accelerated by hardware so that the CPU doesn’t have to process these packets in software.

Network Overlay

Virtual networking overlays are vital for modern data centers, providing flexibility, scalability, and increased security. A virtual overlay network is a computer network built on top of an existing physical network. This abstraction layer creates a virtual overlay solution that can be configured independently of the underlying physical network.

VXLAN multicast mode
Diagram: VXLAN multicast mode

Increased Security

Virtual networks offer many advantages over traditional physical networks. First, they allow for rapidly deploying virtual machines, applications, and services. This can be done in minutes rather than the days or weeks it would take to reconfigure a physical network. Additionally, virtual networks provide increased security, as traffic between VMs can be isolated and encrypted. This can help prevent data breaches and unauthorized access.

Additional Flexibility

Virtual networks are also much more flexible than physical networks. For example, virtual networks can be segmented to separate different types of traffic. This improves performance, as traffic from other applications can be routed separately. Furthermore, virtual networks can be quickly reconfigured to meet changing demands. This agility allows for quick responses to changing business requirements.

Related: Before you proceed, you may find the following useful:

  1. SD-WAN Overlay
  2. Open Networking
  3. Segment Routing
  4. SDN Data Center
  5. Network Overlays
  6. Virtual Switch
  7. Load Balancing
  8. OpenContrail
  9. What is BGP Protocol in Networking



Overlay Virtual Networking

Key Overlay Virtual Network Discussion Points:


  • Introduction to overlay virtual networking and what is involved.

  • Highlighting the details of the virtual overlay solution and the components used.

  • Critical points on the scalability and security concerns.

  • Technical details on the different types of overlays, such as STT and VXLAN.

  • Closing comments on network overlay controllers.

Back to Basics: Virtual networks and virtualization.

Underlay and Overlay Networks

Overlay networks are virtual networks that run on top of physical networks. You have probably seen this terminology even if you have never heard of it. A GRE tunnel can illustrate an overlay network. Physical underlay networks support the GRE tunnel.VXLAN overlays are layer 2 Ethernet networks. Layer 3 IP networks form the underlay network. Transport networks are also known as underlay networks.

Getting packets from A to B is the only job of the underlay network. Layer 2 is not used here, only layer 3. We can load balance traffic on redundant links using an IGP like OSPF or EIGRP.

In addition, the overlay and underlay networks are independent. Underlay networks are virtual, but any changes made to the overlay network won’t affect the underlay network. A routing protocol can reach the destination regardless of how many links you add or remove in the underlay network.

Virtual Networking 

Main Virtual Overlay Networking Components

Overlay Virtual Networks

  • Overlay networks are virtual networks that run on top of physical networks

  • The most common forms of network virtualization are virtual LANs (VLANs), virtual private networks (VPNs), and Multiprotocol Label Switching (MPLS)

  • Like the ACI network, virtual overlay networks work best with Leaf and Spine fabric architectures

  • STT and VXAN can use 5-tuple load balancing as they use port numbers

Virtual overlay solutions

Virtual overlay solutions must have some simple to complex application stacks. Therefore, public or private cloud environments must support austere, complex environments to enable the virtual overlay network. On the other hand, simple customers that require web-hosting solutions need only a single domain with a few segments. Regarding network connectivity, there is one Virtual Machine ( VM ) with a single public IP.

Complex customers require complex multi-tier application stacks with overlay virtual networking, load-balancing, and firewall services in front and between application tiers. Cloud providers must support all types of application stacks as they are isolated virtual segments, and this is done with virtual overlay networks.

Lab guide on VXLAN.

In the following example, we have a lab guide on VXLAN. Here, we created a Layer 2 overlay across the core. The core layer consists of two spines and is a routed layer. The core does not know the subnets assigned to the desktop devices. It is the role of VXLAN to tunnel this information.

Notice we have a VNI set to 6002. This needs to match at both ends of Leaf A and Leaf B. If you change the VNI, you will break connectivity. This is a Layer 2 overlay, as the VNI is mapped to a bridge domain.

VXLAN
Diagram: Changing the VNI

Concept of network virtualization

It’s worth mentioning that network virtualization is nothing new. The most common forms of network virtualization are virtual LANs (VLANs), virtual private networks (VPNs), and Multiprotocol Label Switching (MPLS). VLAN has been the first to extract the location of Layer 2 connectivity across multiple Layer 2 switches. VPN enables overlay networks across untrusted networks such as the WAN, while MPLS segments traffic based on labels.

These technologies enable the administrators to physically separate endpoints into logical groups, making them behave like they are all on the same local (physical) segment. The ability to do this allows for much greater efficiency in traffic control, security, and network management.

    • Enhanced Connectivity:

One of the primary advantages of network overlay is its ability to enhance connectivity. By creating a virtual network layer, overlay networks enable seamless communication between devices and applications, irrespective of their physical location.

This means organizations can effortlessly connect geographically dispersed branches, data centers, and cloud environments, fostering collaboration and resource sharing. Moreover, network overlays offer greater flexibility by allowing organizations to dynamically adjust and optimize their network configurations to meet evolving business needs.

    • Improved Scalability:

Traditional network infrastructures often struggle to keep up with the increasing demands of modern applications and services. Network overlay addresses this challenge by providing a scalable solution. By decoupling the virtual network from the physical infrastructure, overlay networks allow for more efficient resource utilization and easier scaling.

Organizations can easily add or remove network elements without disrupting the entire network. As a result, network overlays enable organizations to scale their networks rapidly and cost-effectively, ensuring optimal performance even during peak usage periods.

Example of an overlay network: MPLS

MPLS overlay is a technique used to create virtual private networks (VPNs) over existing IP networks, enabling organizations to achieve enhanced network scalability, reliability, and security. Unlike traditional IP routing, MPLS overlay relies on labels to forward packets, making it more efficient and flexible.

Overlay with MPLS 

With MPLS, we can have a free BGP core providing an MPLS overlay. MPLS overlay is a network architecture that allows organizations to build virtual private networks (VPNs) on top of their existing network infrastructure. It leverages the capabilities of MPLS technology to create virtual tunnels, known as MPLS tunnels or MPLS paths, which enable the secure and efficient transfer of data between different network endpoints.

Below, we have BGP running between the PEs and carrying customer prefixes for CE 1 and 2. The P, representing the core layer, does not know customer routes and performs label switching. This brings not only scalability, as the P nodes can focus on label switching, but also an added layer of security. No security devices need to be present in the core layer. Although you would need QoS, they are pushing intelligence to the edges.

MPLS forwarding
Diagram: MPLS Overlay

Benefits of MPLS Overlay:

1. Enhanced Performance: MPLS overlay offers improved network performance by enabling faster data transmission and reduced latency. It achieves this by using label switching, which helps prioritize and route data packets efficiently, reducing congestion and optimizing network utilization.

2. Scalability and Flexibility: With MPLS overlay, organizations can quickly expand their network infrastructure without requiring extensive hardware upgrades. It allows for the creation of virtual networks within a shared physical infrastructure, enabling seamless scalability and flexibility.

3. Quality of Service (QoS): MPLS overlay provides enhanced QoS capabilities, enabling organizations to prioritize critical applications or data traffic. This ensures mission-critical applications receive the bandwidth and low latency, optimizing overall network performance.

4. Improved Security: MPLS overlay enhances network security by providing inherent isolation between different VPNs. It creates separate virtual tunnels for each VPN, ensuring that data remains isolated and protected from unauthorized access.

Lab Guide on MPLS TE

In this lab, we will examine MPLS TE with ISIS configuration. Our MPLS core network consists of routers PE1, P1, P2, P3, and PE2. The CE1 and CE2 routers use regular IP routing. All routers are configured to use IS-IS L2. 

MPLS TE is a mechanism that allows network operators to control and manage traffic flows within a Multiprotocol Label Switching (MPLS) network. It is designed to address the limitations of traditional IP routing by providing a more efficient and flexible approach to data forwarding

Note:

There are four main items we have to configure:

  • Enable MPLS TE support:
    • Globally
    • Interfaces
  • Configure IS-IS to support MPLS TE.
  • Configure RSVP.
  • Configure a tunnel interface.
MPLS TE
Diagram: MPLS TE

Example of an overlay network: DMVPN

With the configuration of DMVPN phase 1, we can have a “hub and spoke” topology, where a single hub site acts as the central point for communication, while the other locations, or “spokes,” connect to the hub through virtual tunnels. This topology provides several benefits, including secure communications between spokes, optimized traffic routing, and reduced overhead for managing the network.

DMVPN also supports dynamic routing protocols, such as Open Shortest Path First (OSPF), allowing for dynamic updates to the network topology. This allows for rapid changes in the network, such as adding or removing spokes, without the need to reconfigure the entire network. Additionally, DMVPN supports multicast traffic, allowing the efficient distribution of data and resources to multiple sites simultaneously.

DMVPN
Diagram: DMVPN. Source is techtarget.

Lab guide with DMVPN

In the following lab, we have DMVPM, which creates an overlay network. The hub, which is R1, created an overlay network over the SP router. The SP router represents the WAN; in reality, the number of nodes in the WAN is irrelevant to DMPVN. The overlay is created between R1, R2, and R3, which act as the spokes.

The protocol used in GRE, specifically point-to-point GRE, as we are running DMVPN Phase 1. The Tunneling protocol of mGRE would have been used if we were running DMVPN Phase 3

DMVPN configuration
Diagram: DMVPN Configuration.

Benefits of DMVPN Overlay:

1. Simplified Network Architecture:

Traditional networking often involves complex and static configurations, making it cumbersome to manage and maintain. DMVPN overlay, on the other hand, simplifies network architecture by providing a dynamic and scalable solution. With DMVPN, organizations can establish secure connections between various branch offices, data centers, and remote users, all while leveraging the existing infrastructure. This simplification leads to reduced administrative overhead and improved network agility.

2. Enhanced Flexibility and Scalability:

DMVPN overlay offers unparalleled flexibility and scalability, making it an ideal choice for organizations with dynamic network requirements. As businesses grow and expand, DMVPN allows for the seamless addition of new sites or remote users without requiring extensive configuration changes. Its ability to establish connections on-demand and dynamically allocate resources ensures that network expansion remains hassle-free and cost-effective.

3. Improved Network Performance:

Network performance is crucial for organizations, directly impacting productivity and user experience. DMVPN overlay utilizes multiple paths and load balancing techniques, allowing for efficient utilization of available bandwidth. By optimizing network traffic, DMVPN ensures that applications and services operate smoothly, even during peak usage periods. Moreover, its ability to prioritize critical traffic and dynamically adjust to network conditions further enhances overall performance.

4. Enhanced Security:

Security remains a top concern for organizations, particularly when transmitting sensitive data across networks. DMVPN overlay addresses these concerns by providing robust encryption and authentication mechanisms. By leveraging IPsec protocols, DMVPN ensures that data confidentiality and integrity are maintained, protecting against unauthorized access and potential threats. The inherent security features of DMVPN make it a reliable choice for organizations looking to maintain a secure network environment.

Types of Overlay Networks

1. Virtual Private Networks (VPNs):

VPNs are one of the most common types of overlay networks. They enable secure communication over public networks by creating an encrypted tunnel between the sender and receiver. Individuals and organizations widely use VPNs to protect sensitive data and maintain privacy. Additionally, they allow users to bypass geographical restrictions and access region-restricted content.

2. Software-Defined Networks (SDNs):

In network architecture, SDNs utilize overlay networks to separate the control plane from the data plane. SDNs provide centralized management, flexibility, and scalability by decoupling network control and forwarding functions. Overlay networks in SDNs enable the creation of virtual networks on top of the physical infrastructure, allowing for more efficient resource allocation and dynamic network provisioning.

3. Peer-to-Peer (P2P) Networks:

P2P overlay networks are decentralized systems that facilitate direct communication and file sharing between nodes without relying on a central server. They leverage overlay networks to establish direct connections between peers and enable efficient data distribution. These networks are widely used for content sharing, real-time streaming, and decentralized applications.

4. Content Delivery Networks (CDNs):

CDNs employ overlay networks to optimize content delivery by strategically distributing content across multiple servers in different geographic regions. By bringing content closer to end-users, CDNs reduce latency and improve performance. Overlay networks in CDNs enable efficient content caching, load balancing, and fault tolerance, resulting in faster and more reliable content delivery.

5. Overlay Multicast Networks:

Overlay multicast networks are designed to distribute data to multiple recipients simultaneously efficiently. These networks use overlay protocols to construct multicast trees and deliver data over these trees. Overlay multicast networks benefit applications such as video streaming, online gaming, and live events broadcasting, where data must be transmitted to many recipients in real-time.

Use Cases of Overlay Virtual Networking:

1. Multi-Tenancy:

Overlay virtual networking provides an ideal solution for organizations to segregate their network resources securely. By creating virtual overlays, multiple tenants can coexist on a single physical network infrastructure without interference. This enables service providers and enterprises to offer distinct network environments to customers or departments while ensuring isolation and security.

2. Data Center Interconnect:

Overlay virtual networking enables efficient and scalable data center interconnect (DCI). With traditional networking, interconnecting multiple data centers across geographies can be complex and costly. However, overlay virtual networking simplifies this process by abstracting the underlying physical infrastructure and providing a unified logical network. It allows organizations to seamlessly extend their networks across multiple data centers, enhancing workload mobility and disaster recovery capabilities.

3. Cloud Computing:

Cloud computing heavily relies on overlay virtual networking to deliver agility and scalability. Cloud providers can dynamically provision and manage network resources by leveraging overlay networks, ensuring optimal customer performance and flexibility. Overlay virtual networking enables the creation of virtual networks that are isolated from each other, allowing for secure and efficient multi-tenant cloud environments.

4. Microservices and Containerization:

The rise of microservices architecture and containerization has presented new networking challenges. Overlay virtual networking provides a solution by enabling seamless communication between microservices and containers, regardless of their physical location. It ensures that applications and services can communicate with each other, even across different hosts or clusters, without complex network configurations.

5. Network Segmentation and Security:

Overlay virtual networking enables granular network segmentation, allowing organizations to implement fine-grained security policies. By creating overlay networks, administrators can isolate different workloads, departments, or applications, ensuring each segment has dedicated network resources and security policies. This enhances security by limiting the lateral movement of threats and reducing the attack surface.

Tailored load balancing

Some customers may not require cloud load balancing services provided by the cloud services if they have optimized web delivery by deploying something like Squid or NGINX. Squid is a caching proxy that improves web request response times by caching frequently requested web pages. NGINX ( open source reverse proxy ) is used to load balance Hypertext Transfer Protocol ( HTTP ) among multiple servers.

Example: Traffic flow and the need for a virtual overlay

Traffic would flow to Web servers and trigger application and database requests. Each tier requires different segments, and in large environments, the limitations of using VLANs to create these segments will bring both scalability and performance problems.

This is why we need virtual overlay solutions. These subnets require Layer 3 and sometimes Layer 2 ( MAC ). Layer 2 connectivity might be for high availability services that rely on gratuitous Address Resolution Protocol ( ARP ) between devices or some other non-routable packet that can not communicate over IP. If the packet is not Layer 3 routable, it needs to communicate via Layer 2 VLANs.

Virtual overlay networking
Diagram: Virtual overlay networking and complex application tiers.

Scalability and Security Concerns

The weakest link in a security paradigm is the lowest application in that segment. Make each application an independent tenant so all other applications are unaffected if a security breach or misuse occurs in one application stack.

Designers should always attempt to design application stacks to minimize beachheading, i.e., an attacker compromising one box and using it to jump to another quickly. Public and private clouds should support multi-tenancy with each application stack.

However, scalability issues arise when you deploy each application as an individual segment. For example, customer X’s cloud application requires four segments; 4000 VLANs soon become 1000 applications. Media Access Control ( MAC ) visibility has an entire reach throughout Layer 2 domains.

Some switches support a low count number of MAC addresses. When a switch reaches its MAC limit, it starts flooding packets, increasing network load and consuming available bandwidth that should be used for production services.

…current broadcast domains can support … around 1,000 end hosts in a single bridged LAN of 100 bridges” (RFC 5556 – TRILL)

NIC in promiscuous mode and failure domains

Server administrators configure server NICs in promiscuous mode to save configuration time. NICs in promiscuous mode look at all frames passing even when the frame is not destined for them. Network cards acting in promiscuous mode are essentially the same as having one VLAN spanning the entire domain. Sniffer products set promiscuous modes to capture all data on a link and usually only act in this mode for troubleshooting purposes.

A well-known issue with Layer 2 networks is that they present a single failure domain with extreme scalability and operational challenges. This is related to Layer 2 Spanning Tree Protocol ( STP ); THRILL is also susceptible to broadcast storms and network meltdowns.

The rise of overlay virtual networks

Previously discussed scalability and operational concerns force vendors to develop new data center technologies. One of the most prevalent new technologies is overlay virtual networks, tunneling over IP. An overlay is a tunnel between two endpoints, allowing frames to be transported. The beauty of overlay architectures is that they enable switch table sizes not to increase as the number of hosts attached increases.

Vendors’ Answer: Virtual Overlay Solutions

Diagram: Virtual overlay solutions.

Virtual Overlay Solution: Keep complexity to the edges.

Ideally, we should run virtual networks over IP like SKYPE runs Voice over IP. The recommended design retains complexity at the network’s edge; the IP transport network provides IP transport. A transport network does not need to be a Layer 2 network and can have as many IP subnets and router hops.

All data ( storage, vMotion, user traffic ) traffic becomes an IP application. The concept resembles how Border Gateway Protocol ( BGP ) applies to TCP. End hosts carry out encapsulation and use the network for transport. Again, complexity is at the edge, similar to the Internet. Keeping complexity to the edge makes Layer 3 fabrics efficient and scalable.

VXLAN, STT, and ( NV ) GRE

Numerous encapsulation methods can tunnel over the IP core. This is known as virtual overlay networking and includes VXLAN, STT, and ( NV ) GRE. The main difference between these technologies is the encapsulation method and minor technological differences with TCP offload and load balancing.

virtual overlay solutions
Diagram: Virtual overlay solution.

The Recommended Design: Leaf and Spine.

Like the ACI network, virtual overlay networks work best with Leaf and Spine fabric architectures. Leaf and Spine designs guarantee any two endpoints get equal bandwidth. VMs on the same Top-of-Rack ( ToR ) switch will have access to more bandwidth than if the VM had to communicate across the Spine layer.

Overlay networks assume that the underlying network has a central endpoint. The transport network should avoid oversubscription as much as possible. If security concerns you, you can always place similar VM appliances on dedicated clusters, one type per physical server.

( NV ) GRE, VXLAN, and STT do not have an built-in security features meaning the transport network MUST be secure.

TCP offload, load balancing & scale-out NAT

TCP can push huge segments down the physical NIC and slice the packet into individual TCP segments, improving TCP performance. For example, you can push 10Gbps from a VM with TCP offload. The problem is that NICs only support VLANs and not VXLANs.

NICIRA added another header in front of TCP segments. TCP is embedded in another TCP. Now, you can use the existing NIC to slice the current TCP segment into smaller TCP segments. It is dramatically improving performance.

STT and VXAN

STT and VXAN can use 5-tuple load balancing as they use port numbers. Therefore, traffic sent between a pair of VMs can use more than one link in the network. Unfortunately, not many switches can load balance based on the GRE payload used by NVGRE.

Scale-out NAT is difficult to implement as an asymmetric path is not guaranteed. Furthermore, the shared state is tied to an outside IP address, which limits scale-out options. To scale out effectively, the state has to be spread across all members of the NAT cluster. The new approach uses floating public IP addresses and one-to-one mapping between floating IP and the private IP address inside—there is no state due to the one-to-one mapping.

Distributed layer 2 & layer 3 forwarding  

They distributed Layer 2 forwarding ( data plane ): Most Overlays offer distributed Layer 2 forwarding. VM can be sent to VM in the same segment. The big question is how they distribute MAC to VTEP – some use multicast and traditional Ethernet flooding, while others use control planes. The big question is how scalable is the control plane.

Distributed Layer 3 forwarding ( data plane ): On the other hand, if you have multiple IP subnets between segments ( not layer 2 ), you need to forward between them. The inter-subnet must not be a choke point. If your data center has lots of intra-traffic ( East to West traffic), avoid centralized inter-subnet forwarding, which will quickly become a traffic choke point.

The router will process ARP if you are doing Layer 3 forwarding. But if you are doing a mix of Layer 2 and 3, make sure you can reduce the flooding by intercepting ARP requests and caching ARP replies, known as distributed ARP Caching.

Scale-out control plane 

Initial overlays used multicast and Ethernet-like learning. Now, some vendors are using controller-based overlays. Keep in mind that the controller can now become a scalability bottleneck. However, many vendors, such as Cisco ACI, can scale the controllers and have a quorum.

Efficient controller scalability is seen when controllers do not participate in the data plane ( do not reply to ARP ). This type of controller scales better than controllers that intercept data plane packets and perform data plane activity. So, the data plane will not be affected if a controller is offline. In the early days of Sofware-Defined Networking, this was not the case. If the controller was down, the network was down.

Scale-out controllers 

Attempt to design scale-out controllers by building a cluster of controllers and having some protocol running between them. You now have clear failure domains. For example, controller A looks after VM segment A and Controller B, and control looks after VM segment B. For cloud deployments in multiple locations, deploy multiple controller clusters in each location.

Availability zones

Design availability zones with hierarchical failure domains by splitting infrastructures into regions. Problems arising in one region do not affect all other regions. You have one or more availability zones within an area for physical and logical isolation.

Availability zones limit the impact of a failure in a failure domain. An example of a failure domain could be a VLAN experiencing a broadcast storm. Attempt to determine the span of VLANs across availability zones—define VLANs to one-ToR switch. Never stretch VLANs as you create a single failure domain by merging two zones.

Do not stretch a VLAN across multiple availability zones. This is why we have network overlays in the first place, so we don’t need to stretch VLAN across the data center. For example, VXLAN uses the VNI to differentiate between Layer 2 and Layer 3 traffic over a routed underlay. We can use VXLAN as the overlay network to span large Layer 2 domains over a routed core.

Availability zones
Diagram: Availability zones. The source is cloudconstruct.

Network Overlay Controllers

As a final note on controllers, controller-based SDN networks participate in data planes and perform activities such as MAC learning and ARP replies. As mentioned, this is not common nowadays but was at the start of the SDN days. If the controller performs activities such as MAC learning and APR replies and the controller fails, then you have network failure.

The more involved the controller is in the forwarding decisions, the worse the outage can be. All overlay networking vendors nowadays have controllers that set up the control plane so the data plane can forward traffic without getting involved in data plane activity. This design also allows the controller to be scaled without affecting the data plane activity.

Overlay virtual networking has significant implications for modern network architectures. It enables the creation of software-defined networks (SDNs), where network policies, routing, and security are managed centrally through software-based controllers. This centralized management simplifies network operations, improves agility, and enables network automation.

Recap: Understanding Overlay Virtual Networking

Overlay virtual networking is a method that allows virtual networks to be created on top of existing physical networks. By decoupling the network’s logical and physical infrastructure, overlay virtual networking provides flexibility, scalability, and enhanced security.

Benefits of Overlay Virtual Networking

Overlay virtual networking brings numerous advantages to organizations. Firstly, it enables seamless network scalability, allowing businesses to quickly expand their network resources without needing physical infrastructure upgrades. Additionally, overlay virtual networking enhances network security by providing isolated virtual networks that can be customized with specific security policies.

Implementation of Overlay Virtual Networking

Implementing overlay virtual networking involves utilizing software-defined networking (SDN) technologies. Organizations can create virtual networks independent of the underlying physical infrastructure through SDN controllers and network virtualization overlays. This implementation approach simplifies network management and enables dynamic configuration.

Applications of Overlay Virtual Networking

Overlay virtual networking finds applications in various industries. In data centers, it facilitates efficient resource allocation and workload mobility across virtual machines. For cloud service providers, overlay virtual networking enables the creation of virtual private clouds, ensuring secure and isolated connectivity for their customers. Moreover, overlay virtual networking can enhance the connectivity and security of IoT devices in smart cities and industrial environments.

Overlay virtual networking empowers organizations to unlock new network flexibility, scalability, and security levels. By abstracting the network’s logical layer from the physical infrastructure, overlay virtual networking enables seamless expansion, simplified management, and tailored security policies. As the digital landscape continues to evolve, overlay virtual networking will play a pivotal role in shaping the future of networking.

Summary: Overlay Virtual Networks

Overlay networking has revolutionized the way we design and manage modern networks. In this blog post, we will delve into the fascinating world of overlay networking, exploring its benefits, applications, and critical components.

Understanding Overlay Networking

Overlay networking is a technique for creating virtual networks on top of an existing physical network infrastructure. By decoupling the network services from the underlying hardware, overlay networks provide flexibility, scalability, and enhanced security.

Benefits of Overlay Networking

One of the primary advantages of overlay networking is its ability to abstract the underlying physical infrastructure, allowing for seamless integration of different network technologies and protocols. This flexibility empowers organizations to adapt to changing network requirements without significant disruptions. Additionally, overlay networks facilitate the implementation of advanced network services, such as virtual private networks (VPNs) and load balancing, while maintaining a simplified management approach.

Applications of Overlay Networking

Overlay networking finds applications in various domains, ranging from data centers to cloud computing. In data center environments, overlay networks enable efficient multi-tenancy, allowing different applications or departments to operate within isolated virtual networks. Moreover, overlay networking facilitates the creation of hybrid cloud architectures, enabling seamless connectivity between on-premises infrastructure and public cloud resources.

Key Components of Overlay Networking

Understanding overlay networking’s key components is crucial to comprehending it. These include overlay protocols, which establish and manage virtual network connections, and software-defined networking (SDN) controllers, which orchestrate the overlay network. Additionally, virtual tunnel endpoints (VTEPs) play a vital role in encapsulating and decapsulating network packets, ensuring efficient communication within the overlay network.

Overlay networking has genuinely transformed the landscape of modern network architectures. By providing flexibility, scalability, and enhanced security, overlay networks have become indispensable in various industries. Whether it is for data centers, cloud environments, or enterprise networks, overlay networking offers a powerful solution to meet the evolving demands of the digital era.

Conclusion:

In conclusion, overlay networking has emerged as a game-changer in the world of networking. Its ability to abstract and virtualize network services brings immense value to organizations, enabling them to adapt quickly, enhance security, and optimize resource utilization. As technology continues to advance, overlay networking will likely play an even more significant role in shaping the future of network architectures.

cloud data center

Cloud Data Center | Modular building blocks

Cloud Data Centers

In today's digital age, where data is generated and consumed at an unprecedented rate, the need for efficient and scalable data storage solutions has become paramount. Cloud data centers have emerged as a groundbreaking technology, revolutionizing the way businesses and individuals store, process, and access their data. This blog post delves into the world of cloud data centers, exploring their inner workings, benefits, and their impact on the digital landscape.

Cloud data centers, also known as cloud computing infrastructures, are highly specialized facilities that house a vast network of servers, storage systems, networking equipment, and software resources. These centers provide on-demand access to a pool of shared computing resources, enabling users to store and process their data remotely. By leveraging virtualization technologies, cloud data centers offer unparalleled flexibility, scalability, and cost-effectiveness.

Scalability and Elasticity: One of the most significant advantages of cloud data centers is their ability to quickly scale resources up or down based on demand. This elastic nature allows businesses to efficiently handle fluctuating workloads, ensuring optimal performance and cost-efficiency.

Cost Savings: Cloud data centers eliminate the need for upfront investments in hardware and infrastructure. Businesses can avoid the expenses associated with maintenance, upgrades, and physical storage space. Instead, they can opt for a pay-as-you-go model, where costs are based on usage, resulting in significant savings.

Enhanced Reliability and Data Security: Cloud data centers employ advanced redundancy measures, including data backups and geographically distributed servers, to ensure high availability and minimize the risk of data loss. Additionally, they implement robust security protocols to safeguard sensitive information, protecting against cyber threats and unauthorized access.

Enterprise Solutions: Cloud data centers offer a wide range of enterprise solutions, including data storage, virtual machine provisioning, software development platforms, and data analytics tools. These services enable businesses to streamline operations, enhance collaboration, and leverage big data insights for strategic decision-making.

Cloud Gaming and Streaming:The gaming industry has witnessed a transformative shift with the advent of cloud data centers. By offloading complex computational tasks to remote servers, gamers can enjoy immersive gaming experiences with reduced latency and improved graphics. Similarly, cloud data centers power streaming platforms, enabling users to access and enjoy high-quality multimedia content on-demand.

Cloud data centers have transformed the way we store, process, and access data. With their scalability, cost-effectiveness, and enhanced security, they have become an indispensable technology for businesses and individuals alike. As we continue to generate and rely on vast amounts of data, cloud data centers will play a pivotal role in driving innovation, efficiency, and digital transformation across various industries.

Highlights: Cloud Data Centers

Data is distributed

Data and applications are being accessed by a multidimensional world of data and applications as our workforce shifts from home offices to centralized campuses to work-from-anywhere setups. Data is widely distributed across on-premises, edge clouds, and public clouds, and business-critical applications are becoming containerized microservices. Agile and resilient networks are essential for providing the best experience for customers and employees.

The IT department faces a multifaceted challenge in synchronizing applications with networks. An automation tool set is essential to securely manage and support hybrid and multi-cloud data center operations. Automation toolsets are also necessary with the growing scope of NetOps and DevOps roles.

Understanding Pod Data Centers

Pod data centers are modular and self-contained units that house all the necessary data processing and storage components. Unlike traditional data centers requiring extensive construction and physical expansion, pod data centers are designed to be easily deployed and scaled as needed. These prefabricated units consist of server racks, power distribution systems, cooling mechanisms, and network connectivity, all enclosed within a secure and compact structure.

The adoption of pod data centers offers several advantages. Firstly, their modular nature allows for rapid deployment and easy scalability. Organizations can quickly add or remove pods based on their computing needs, resulting in cost savings and flexibility. Additionally, pod data centers are highly energy-efficient, incorporating advanced cooling techniques and power management systems to optimize resource consumption. This not only reduces operational costs but also minimizes the environmental impact.

source: TechTarget

 

Enhanced Reliability and Redundancy

Pod data centers are designed with redundancy in mind. Organizations can ensure high availability and fault tolerance by housing multiple pods within a facility. In the event of a hardware failure or maintenance, the workload can be seamlessly shifted to other functioning pods, minimizing downtime and ensuring uninterrupted service. This enhanced reliability is crucial for industries where downtime can lead to significant financial losses or compromised data integrity.

The rise of pod data centers has paved the way for further innovations in computing infrastructure. As the demand for data processing continues to grow, pod data centers will likely become more compact, efficient, and capable of handling massive workloads. Additionally, advancements in edge computing and the Internet of Things (IoT) can further leverage the benefits of pod data centers, bringing computing resources closer to the source of data generation and reducing latency.

Data center network virtualization

Network Virtualization of networks plays a significant role in designing data centers, especially those for use in the cloud space. There is not enough space here to survey every virtualization solution proposed or deployed (such as VXLAN, nvGRE, MPLS, and many others); a general outline of why network virtualization is essential will be considered in this section.

A primary goal of these technologies is to move the control plane state from the core to the network’s edges. With VXLAN, a Layer 3 fabric can be used to build Layer 2 broadcast domains. For each ToR, spine switches only know a few addresses, reducing the state carried in the IP routing control plane to a minimum.

what is spine and leaf architecture

Tunneling will affect visibility to quality of service and other traffic segregation mechanisms within the spine or the data center core, which is the first question relating to these technologies. In theory, tunneling traffic edge-to-edge could significantly reduce the state held at spine switches (and perhaps even at ToR switches). Still, it could sacrifice fine-grained control over packet handling.

Tunnel Termination

In addition, where should these tunnels be terminated? The traffic flows across the fabric can be pretty exciting if they are terminated in software running on the data center’s compute resources (such as in a user VM space, the software control space, or hypervisor space). In this case, traffic is threaded from one VLAN to another through various software tunnels and virtual routing devices. However, the problem of maintaining and managing hardware designed to support these tunnels can still exist if these tunnels terminate on either the ToR or in the border leaf nodes.

VXLAN unicast mode

Modular Data Center Design

A modular data center design consists of several prefabricated modules or a deployment method for delivering data center infrastructure in a modular, quick, and flexible process. The modular building block design approach is necessary for large data centers as “Hugh domains fail for a reason” – “Russ White.” For the virtual data center, these modular building blocks can be referred to as “Points of Delivery,” also known as pods, and “Integrated Compute Stacks,” also known as ICSs, such as VCE Vblock and FlexPod.

Example: Cisco ACI 

You could define a pod as a modular unit of data center components ( pod data center ) that support incremental build-out of the data center. They are the basis for modularity within the cloud data center and are the basis of design in the ACI network. Based on spine leaf architecture. To scale a pod and expand incrementally, designers can add Integrated Compute Stacks ( ICS ) within a pod. ICS is a second, smaller unit added as a repeatable unit.

Before you proceed, you may find the following posts helpful:

  1. Container Networking
  2. OpenShift Networking
  3. OpenShift SDN
  4. Kubernetes Networking 101
  5. OpenStack Architecture

Modular data center design.

Key Modular Building Blocks Discussion Points:


  • Introduction to Modular Building Blocks and what is involved.

  • Highlighting the details of a modular data center design.

  • Critical points on the use of POD and how to build a POD data center.

  • A final note on designing for multi-tenancy.

Back to Basics with a data center design

Data centers were significantly dissimilar from those just a short time ago. Infrastructure has moved from traditional on-premises physical servers to virtual networks. These virtual networks must seamlessly support applications and workloads across physical infrastructure pools and multi-cloud environments. Generally, a data center consists of the following core infrastructure components: network infrastructure, storage infrastructure, and compute infrastructure.

Modular Data Center Design

Scalability:

One key advantage of cloud data centers is their scalability. Unlike traditional data centers, which require physical infrastructure upgrades to accommodate increased storage or processing needs, cloud data centers can quickly scale up or down based on demand. This flexibility allows businesses to adapt rapidly to changing requirements without incurring significant costs or disruptions to their operations.

Efficiency:

Cloud data centers are designed to maximize energy consumption and hardware utilization efficiency. By consolidating multiple servers and storage devices into a centralized location, cloud data centers reduce the physical footprint required to store and process data. This minimizes the environmental impact and helps businesses save on space, power, and cooling costs.

Reliability:

Cloud data centers are built with redundancy in mind. They have multiple power sources, network connections, and backup systems to ensure uninterrupted service availability. This high level of reliability helps businesses avoid costly downtime and ensures that their data is always accessible, even in the event of hardware failures or natural disasters.

Security:

Data security is a top priority for businesses, and cloud data centers offer robust security measures to protect sensitive information. These facilities employ various security protocols such as encryption, firewalls, and intrusion detection systems to safeguard data from unauthorized access or breaches. Cloud data centers often comply with industry-specific regulations and standards to ensure data privacy and compliance.

Cost Savings:

Cloud data centers offer significant cost savings compared to maintaining an on-premises data center. With cloud-based infrastructure, businesses can avoid upfront capital expenditures on hardware and maintenance costs. Instead, they can opt for a pay-as-you-go model, where they only pay for the resources they use. This scalability and cost efficiency make cloud data centers attractive for businesses looking to reduce IT infrastructure expenses.

The general idea behind these two forms of modularity is to have consistent, predictable configurations with supporting implementation plans that can be rolled out when a predefined performance limit is reached. For example, if pod-A reaches 70% capacity, a new pod called pod-B is implemented precisely. The critical point is that the modular architecture provides a predictable set of resource characteristics that can be added as needed. This adds numerous benefits to fault isolation, capacity planning, and ease of new technology adoption. Special service pods can be used for specific security and management functions.

pod data center
Diagram: The pod data center and modularity.

Pod Data Center

No two data centers will be the same with all the different components. However, a large-scale data center will include key elements: applications, servers, storage, networking such as load balancers, and other infrastructure. These can be separated into different pods. A pod is short for Performance Optimized Datacenter and has been used to describe several different data center enclosures. Most commonly, these pods are modular data center solutions with a single-aisle, multi-rack enclosure with built-in hot- or cold-aisle containment.

A key point: Pod size

The pod size is relative to the MAC addresses supported at the aggregation layer. Different vNICs require unique MAC addresses, usually 4 MAC addresses per VM. For example, the Nexus 7000 series supports up to 128,000 MAC addresses, so in a large POD design, 11,472 workloads can be enabled, translating to 11,472 VM – 45,888 MAC addresses. Sharing VLANS among different pods is not recommended, and you should try to filter VLANs on trunk ports to stop unnecessary MAC address flooding. In addition, spanning VLANs among PODs would result in an end-to-end spanning tree, which should be avoided at all costs.

Pod data center and muti-tenancy

Within these pods and ICS stacks, multi-tenancy and tenant separation is critical. A tenant is an entity subscribing to cloud services and can be defined in two ways. First, a tenant’s definition depends on its location in the networking world. For example, a tenant in the private enterprise cloud could be a department or business unit. However, a tenant in the public world could be an individual customer or an organization.

Each tenant can have differentiating levels of resource allocation within the cloud. Depending on the requirements, cloud services can range from IaaS to PaaS, ERP, SaaS, and more. Standard service offerings fall into four tiers: Premium, Gold, Silver, and Bronze. In addition, recent tiers, such as Copper and Palladium, will be discussed in later posts.

It does this by selecting a network container that provides them with a virtual dedicated network ( within a shared infrastructure ). The customer then goes through a VM sizing model, storage allocation/protection, and the disaster recovery tier.

Modular building blocks
Modular building blocks and service tiers.

Example of a tiered service model

Component

Gold

Silver 

Bronze

Segmentation

Single VRF

Single VRF

Single VRF

Data recovery

Remote replication

Remote replicaton

None

VLAN

Mulit VLAN

Multi VLAN

Single VLAN

Service

FW and LB service

LB service

None

Data protection

Clone

Snap

None

Bandwidth

40%

30% 

20%

Modular building blocks: Network container

The type of service selected in the network container will vary depending on application requirements. In some cases, applications may require several tiers. For example, a Gold tier could require a three-tier application layout ( front end, application, and database ). Each tier is placed on a separate VLAN, requiring stateful services ( dedicated virtual firewall and load balancing instances). Other tiers may require a shared VLAN with front-end firewalling to restrict inbound traffic flows.

Usually, a tier will use a single individual VRF ( VRF-lite ), but the number of VLANs will vary depending on the service level. For example, a cloud provider offering simple web hosting will provide a single VRF and VLAN. On the other hand, an enterprise customer with a multi-layer architecture may want multiple VLANs and services ( load balancer, Firewall, Security groups, cache ) for its application stack.

Modular building blocks: Compute layer

The compute layer is related to the virtual servers and the resources available to the virtual machines. Service profiles can vary depending on the size of the VM attributes, CPU, memory, and storage capacity. Service tiers usually have three compute workload sizes at a compute layer, as depicted in the table below.

Pod data center: Example of computing resources

Component

Large

Medium

Small

vCPU per VM

 1 vCPU

0.5 vCPU

 0.25 vCPU

Cores per CPU

4

4

4

VM per CPU

4 VM

16 VM

32 VM

VM per vCPU oversubscription

1:1 ( 1 )

2:1 ( 0.5 )

4:1 ( 0.25 )

RAM allocation

16 GB dedicated 

8 GB dedicated

4 GB shared

Compute profiles can also be associated with VMware Distributed Resource Scheduling ( DRS ) profiles to prioritize specific classes of VMs.

Modular building blocks: Storage Layer

This layer relates to storage allocation and the type of storage protection. For example, a GOLD tier could offer three tiers of RAID-10 storage using 15K rpm FC, 10K rpm FC, and SATA drives. While a BRONZE tier could offer just a single RAID-5 with SATA drives

Summary: Cloud Data Centers

In the rapidly evolving digital age, data centers play a crucial role in storing and processing vast amounts of information. Traditional data centers have long been associated with high costs, inefficiencies, and limited scalability. However, a new paradigm has emerged – modular data center design. This innovative approach offers many benefits, revolutionizing how we think about data centers. This blog post explored the fascinating world of modular data center design and its impact on the industry.

Understanding Modular Data Centers

Modular data centers, also known as containerized data centers, are self-contained units that house all the essential components required for data storage and processing. These pre-fabricated modules are built off-site and can be easily transported and deployed. The modular design encompasses power and cooling systems, racks, servers, networking equipment, and security measures. This plug-and-play concept allows for rapid deployment, flexibility, and scalability, making it a game-changer in the data center realm.

Benefits of Modular Data Center Design

Scalability and Flexibility

One key advantage of modular data center design is its scalability. Traditional data centers often face challenges in accommodating growth or adapting to changing needs. However, modular data centers offer the flexibility to scale up or down by simply adding or removing modules as required. This modular approach allows organizations to seamlessly align their data center infrastructure with their evolving business demands.

Cost Efficiency

Modular data center design brings notable cost advantages. Traditional data centers often involve significant upfront investments in construction, power distribution, cooling infrastructure, etc. In contrast, modular data centers reduce these costs by utilizing standardized modules that are pre-engineered and pre-tested. Additionally, scalability ensures that organizations only invest in what they currently need, avoiding unnecessary expenses.

Rapid Deployment

Time is of the essence in today’s fast-paced world. Traditional data centers can design, build, and deploy for months or even years. On the other hand, modular data centers can be rapidly deployed within weeks, thanks to their pre-fabricated nature. This accelerated deployment allows organizations to meet critical deadlines, swiftly respond to market demands, and gain a competitive edge.

Enhanced Efficiency and Performance

Optimized Cooling and Power Distribution

Modular data centers are designed with efficiency in mind. They incorporate advanced cooling technologies, such as hot and cold aisle containment, precision cooling, and efficient power distribution systems. These optimizations reduce energy consumption, lower operational costs, and improve performance.

Simplified Maintenance and Upgrades

Maintaining and upgrading traditional data centers can be a cumbersome and disruptive process. Modular data centers simplify these activities by providing a modularized framework. Modules can be easily replaced or upgraded without affecting the entire data center infrastructure. This modularity minimizes downtime and ensures continuous operations.

Conclusion:

In conclusion, modular data center design represents a significant leap forward in data centers. Its scalability, cost efficiency, rapid deployment, and enhanced efficiency make it a compelling choice for organizations looking to streamline their infrastructure. As technology continues to evolve, modular data centers offer the flexibility and agility required to meet the ever-changing demands of the digital landscape. Embracing this innovative approach will undoubtedly shape the future of data centers and pave the way for a more efficient and scalable digital infrastructure.