Design Factors for Data Center Networks

When designing a data center network, network professionals must consider factors unrelated to their area of specialization. To avoid a network topology becoming a bottleneck for expansion, a design must consider the data center’s growth rate (expressed as the number of servers, switch ports, customers, or any other metric). Data center network designs must also consider application bandwidth demand. Network professionals commonly use the oversubscription concept to translate such demand into more relatable units (such as ports or switch modules).

Oversubscription

Oversubscription occurs when multiple elements share a common resource and the allocated resources per user exceed the maximum value that each can use. Oversubscription refers to the amount of bandwidth switches can offer downstream devices at each layer in data center networks. The ratio of upstream server traffic oversubscription at the access layer switch would be 4:1, for example, if it has 32 10 Gigabit Ethernet server ports and eight uplink 10 Gigabit Ethernet interfaces.

Sizing Failure Domains

Oversubscription ratios must be tested and fine-tuned to determine the optimal network design for the application’s current and future needs.

Business-related decisions also influence the failure domain sizing of a data center network. The number of servers per IP subnet, access switch, or aggregation switch may not be solely determined by technical aspects if an organization cannot afford to lose multiple application environments simultaneously.

Data center network designs are affected by application resilience because they require perfect harmony between application and network availability mechanisms. An example would be:

• An active server connection should be connected to an isolated network using redundant Ethernet interfaces.
• An application server must be able to respond faster to a connection failure than the network.

Last, a data center network designer must be aware of situations where all factors should be prioritized since benefiting one aspect could be detrimental to another. Traditionally, the topology between the aggregation and access layers illustrates this situation.

Gaining Efficiency

Deploying multiple tenants on a shared infrastructure is far more efficient than having single tenants per physical device. With a virtualized infrastructure, each tenant requires isolation from all other tenants sharing the same physical infrastructure.

For a data center network design, each network container requires path isolation, for example, 802.1Q on a shared Ethernet link between two switches, and device virtualization at the different network layers, for example, Cisco Application Control Engine ( ACE ) or Cisco Firewall Services Module ( FWSM ) virtual context. To implement independent paths with this type of data center design, you can create Virtual Routing Forwarding ( VRF ) per tenant and map the VRF to Layer 2 segments.

Example: Virtual Data Center Design. Cisco.

More recently, the Cisco ACI network enabled segmentation based on logical security zones known as endpoint groups, where security constructs known as contracts are needed to communicate between endpoint groups. The Cisco ACI still uses VRFs, but they are used differently. Then, we have the Ansible Architecture, which can be used with Ansible variables to automate the deployment of the network and security constructs for the virtual data center. This brings consistency and will eliminate human error.

Before you proceed, you may find the following posts helpful for pre-information:

1. Context Firewall
2. Virtual Device Context
3. Dynamic Workload Scaling
4. ASA Failover
5. Data Center Design Guide

Back to basics with data center types.

Numerous kinds of data centers and service models are available. Their category relies on several critical criteria. Such as whether one or many organizations own them, how they serve in the topology of other data centers, and what technologies they use for computing and storage. The main types of data centers include:

• Enterprise data centers.
• Managed services data centers.
• Colocation data centers.
• Cloud data centers.

You may build and maintain your own hybrid cloud data centers, lease space within colocation facilities, also known as colos, consume shared compute and storage services, or even use public cloud-based services.

Benefits of Virtual Data Centers:

1. Scalability: Virtual data centers offer unparalleled scalability, allowing businesses to expand or contract their infrastructure quickly based on evolving needs. With the ability to provision additional resources in real time, organizations can quickly adapt to changing workloads, ensuring optimal performance and reducing downtime.

2. Cost Efficiency: Virtual data centers significantly reduce operating costs by eliminating the need for physical servers and reducing power consumption. Consolidating multiple VMs onto a single physical server optimizes resource utilization, improving cost efficiency and lowering hardware requirements.

3. Flexibility: Virtual data centers allow organizations to deploy and manage applications across multiple cloud platforms or on-premises infrastructure. This hybrid cloud approach enables seamless workload migration, disaster recovery, and improved business continuity.

Critical Components of Virtual Data Centers:

1. Hypervisor: At the core of a virtual data center lies the hypervisor, a software layer that partitions physical servers into multiple VMs, each running its operating system and applications. Hypervisors enable the efficient utilization of hardware resources and facilitate VM management.

2. Software-Defined Networking (SDN): SDN allows organizations to define and manage their network infrastructure through software, decoupling network control from physical devices. This technology enhances flexibility, simplifies network management, and enables greater security and agility within virtual data centers.

3. Virtual Storage: Virtual storage technologies, such as software-defined storage (SDS), enable the pooling and abstraction of storage resources. This approach allows for centralized management, improved data protection, and simplified storage provisioning in virtual data centers.

Data center network design: VRF-lite

VRF information from a static or dynamic routing protocol is carried across hop-by-hop in a Layer 3 domain. Multiple VLANs in the Layer 2 domain are mapped to the corresponding VRF. VRF-lite is known as a hop-by-hop virtualization technique. The VRF instance logically separates tenants on the same physical device from a control plane perspective.

From a data plane perspective, the VLAN tags provide path isolation on each point-to-point Ethernet link that connects to the Layer 3 network. VRFs provide per-tenant routing and forwarding tables and ensure no server-server traffic is permitted unless explicitly allowed.

Service Modules in Active/Active Mode

Multiple virtual contexts

The service layer must also be virtualized for tenant separation. The network services layer can be designed with a dedicated Data Center Services Node ( DSN ) or external physical appliances connected to the core/aggregation. The Cisco DSN data center design cases use virtual device contexts (VDC), virtual PortChannel (vPC), virtual switching system (VSS), VRF, and Cisco FWSM and Cisco ACE virtualization. 

This post will look at a DSN as a self-contained Catalyst 6500 series with ACE and firewall service modules. Virtualization at the services layer can be accomplished by creating separate contexts representing separate virtual devices. Multiple contexts are similar to having multiple standalone devices.

The Cisco Firewall Services Module ( FWSM ) provides a stateful inspection firewall service within a Catalyst 6500. It also offers separation through a virtual security context that can be transparently implemented as Layer 2 or as a router “hop” at Layer 3. The Cisco Application Control Engine ( ACE ) module also provides a range of load-balancing capabilities within a Catalyst 6500.

With a context design, you can offer high availability and efficient load distribution. The first FWSM and ACE are primary for the first context and standby for the second context. The second FWSM and ACE are primary for the second context and standby for the first context. Traffic is not automatically load-balanced equally across the contexts. Additional configuration steps are needed to configure different subnets in specific contexts.

Compute separation

Traditional security architecture placed the security device in a central position, either in “transparent” or “routed” mode. Before communication could occur, all inter-host traffic had to be routed and filtered by the firewall device located at the aggregation layer. This works well in low-virtualized environments when there are few VMs. Still, a high-density model ( heavily virtualized environment ) forces us to reconsider firewall scale requirements at the aggregation layer.

It is recommended that virtual firewalls be deployed at the access layer to address the challenge of VM density and the ability to move VMs while keeping their security policies. This creates intra and inter-tenant zones and enables finer security granularity within single or multiple VLANs.

Application tier separation

The Network-Centric model relies on VLAN separation for three-tier application deployment for each tier. Each tier should have its VLAN in one VRF instance. If VLAN-to-VLAN communication needs to occur, traffic must be routed via a default gateway where security policies can enforce traffic inspection or redirection.

vShield ( vApp ) virtual appliance can inspect inter-VM traffic among ESX hosts, and layers 2,3,4, and 7 filters are supported. A drawback of this approach is that the FW can become a choke point. Compared to the Network-Centric model, the Server-Centric model uses separate VM vNICs and daisy chain tiers.

 Data center network design with Security Groups

The concept of Security groups replacing subnet-level firewalls with per-VM firewalls/ACLs. With this approach, there is no traffic tromboning or single choke points. It can be implemented with Cloudstack, OpenStack ( Neutron plugin extension ), and VMware vShield Edge. Security groups are elementary; you assign VMs and specify filters between groups. 

Security groups are suitable for policy-based filtering but don’t consider other functionality where data plane states are required for replay attacks. Security groups give you echo-based functionality, which should be good enough for current TCP stacks that have been hardened over the last 30 years. But if you require full stateful inspection and do not regularly patch your servers, then you should implement a complete stateful-based firewall.