Data centers and modularity

There are two ways to approach modularity in data center design. Each leaf (pod or rack) must be constructed entirely in the first step. Each pod contains the necessary storage, processing, and other services to perform a specific task. It is possible to design pods to provide Hadoop databases and human resources systems or even build application environments.

In a modular network, pods can be exchanged relatively independently of each other and other services and pods. Services can be connected (or disconnected) according to their needs. This model is extremely flexible and ideal for enterprises and other users of data centers with rapidly changing needs.

The second approach modularizes pods according to their resource availability. Block storage pods, file storage pods, virtualized compute pods, and bare metal compute pods can all be housed in different pods. The network operator can minimize the effect of upgrading one type of resource on the operation of specific services in the data center by upgrading it in bulk. This solution would benefit organizations that virtualize most of their services on standard hardware and want to separate hardware and software lifecycle management.

Of course, the two options can be mixed. In a data protection pod, backup services might be provided to other pods, which would then be organized based on their services rather than their resources. A resource-based modularization plan may be interrupted if an occasional service runs on bare metal servers instead of virtual servers. There are two types of traffic in these situations: those that can be moved for optimal traffic levels and those that cannot.

Performing modularization

With virtualization modularization, systems are deemed modular when they can be decomposed into several components that may be mixed and matched in various configurations. So, with virtualization modularization, we don’t have one flat network; we have different modules with virtualization as the base technology performing the modularization. Some of these virtualization technologies include MPLS.

Overlay Networking: Modular Partitions

To move data across the physical network, overlay services, and data-plane encapsulations must be defined. Underlay networks (or simply underlays) are typically used for this type of transport. The OSI layer at which tunnel encapsulation occurs is crucial to determining the underlay. The overlay header type somewhat dictates the transport network type. With VXLAN, for example, the underlying transport network (underlay) is a Layer 3 network that transports VXLAN-encapsulated packets between the source and destination tunnel edge devices. As a result, the underlay facilitates reachability between the tunnel edge devices and the overlay edge devices.

Reducing state and control plane

Why don’t we rebuild the Internet into one flat-switched domain – the flat earth model? The problem with designing one significant flat architecture is that you would find no way to reduce individual devices’ state and control plane. To forward packets efficiently, every device would have to know how to reach every other device; each device would also have to be interrupted every time there was a state change on any router in the entire domain. This is in contrast to modularization virtualization, also called virtualization modularization.

Modularity: Data Center Design

Modularity in data center design can be approached in two ways.

To begin with, each leaf (or pod, or “rack”) should be constructed as a complete unit. Each pod provides storage, processing, and other services to perform all the tasks associated with one specific service set. One pod may be designed to process and store Hadoop data, another for human resources management, or an application build environment.

This modularity allows the network designer to interchange different types of pods without affecting other pods or services in the network. By connecting (or disconnecting) services as needed, the fabric becomes a “black box”. The model is flexible for enterprises and other data center users whose needs constantly change.

In addition, pods can be modularized according to the type of resources they offer. The bare metal compute, the virtualized compute, and the block storage pods may be housed in different pods. As a result, the network operator can upgrade one type of resource en masse with minimal impact on the operation of any particular service in the data center. A solution like this is more suited to organizations that can virtualize most of their services onto standard hardware and want to manage the hardware life cycle separately from the software life cycle.

Related: Before you proceed, you may find the following posts helpful:

1. What is VXLAN
2. Container Based Virtualization
3. What is Segment Routing
4. WAN Virtualization
5. WAN SDN
6. IPSec Fault Tolerance

Back to basics with network modularity and hierarchical network design

Hierarchical network design reaches beyond hub-and-spoke topologies at the module level and provides rules, or general design methods, that give the best overall network design. 

The first rule is to assign each module a single function. Reducing the number of functions or roles assigned to any particular module will help. It will also streamline the configuration of devices within the module and along its edge. 

The second general rule in the hierarchical method is to design the network modules. Hence, every module at a given layer or distance from the network core has a roughly parallel function.

The amount of state and the rate at which it changes is impossible to maintain, and what you would witness would be a case of information overload at the machine level. Machine overload can be diagnosed into three independent problems below. The general idea behind machine overload is that too much information is insufficient for network efficiency. Some methods can reduce these defects, but no matter how much you try to optimize your design, you will never get away from the fact that fewer routes in a small domain are better than many routes in a large domain.

CPU and memory utilization

On most Catalyst platforms, routing information is stored in a special high-speed memory called TCAM. Unfortunately, TCAM is not infinite and is generally expensive. Large routing tables require more CPU cycles, physical memory, and TCAM.

Rate of state of change

Every time the network topology changes, the control plane must adapt to the new topology. The bigger the domain, the more routers will have to recalculate the best path and propagate changes to their neighbors, increasing the rate of state change. Because MAC addresses are not hierarchical, a Layer 2 network has a much higher rate of state change than a Layer 3 network.

Positive feedback loops

Positive feedback loops add the concept of rate of change with the rate of information flow.

How can we address these challenges? The answer is network design with modularization and information hiding with the technique of virtualization modularization.

Modularization, virtualization, and information hiding

Information hiding reduces routing table sizes and state change rates by combining multiple destinations into one summary prefix, aggregation, or separating destinations into sub-topologies, aka virtualization. Information hiding can also be carried out by configuring route filters at specific network points.

Router B summarizes network 192.168.0.0/16 in the diagram below and sends the aggregate route to Router C. The aggregation process hides more specific routes behind Router A. Router C never receives any specifics or state changes for those specifics, so it doesn’t have to do any recalculations if the reachability of those networks changes. Link flaps and topology changes on Router A will not be known to Router C and vice versa.

Positive feedback loops

Positive feedback loops add the concept of rate of change with the rate of information flow.

Routers A and B are also in separate failure domains from router C. Routers C’s view of the network differs from Routers A and B. A failure domain is the set of devices that must recalculate their control plane information in the case of a topology change.

When a link or node fails in one fault domain, it does not affect the other. There is an actual split in the network. You could argue that aggregation does not split the network into “true” fault domains, as you can still have backup paths ( specific routes ) with different metrics reachable in the other domain.

If we split the network into fault domains, devices within each fault domain only compute paths within their fault domain. This drags the network closer to the MTTR/MTBF balance point, which is another reason you should divide complexity from complexity.

Virtualization Modularization

The essence of network design and fault domain isolation is based on the modularization principle. Modularization breaks up the control plane, giving you different information in different network sections. It would help if you engineered the network so it can manage organic growth and change with fixed limits. You can move to the next module when the network gets too big. The concept of repeatable configurations creates a more manageable network. Each topology should be designed and configured using the same tools where possible. 

Why Modularize?

The prime reason to introduce modularity and a design with modular building blocks is to reduce the amount of data any particular network device must handle when it describes and calculates paths to a specific destination. The less information the routing process has to process, the faster the network will converge in conjunction with tight modulation limits.

The essence of modularization can be traced back to why the OSI and TCP/IP models were introduced. So why do we have these models? First, they allow network engineers to break big problems into little pieces so we can focus on specific elements and not get clouded by the complexity of the entire problem all at once. With the practice of modulation, particular areas of the network are assigned specific tasks.

The core focuses solely on fast packet forwarding, while the edge carries out various functions such as policing, packet filtering, QoS classification, etc. Modulization is done by assigning specific tasks to different points in the network.

Virtualization techniques to perform modularization

Virtualization techniques such as MPLS and 802.1Q are also ways to perform modularization. The difference is that they are vertical rather than horizontal. Virtualization can be thought of as hiding information and vertical layers within a network. So why don’t we perform modularization on every router and put each router into a single domain? The answer is network stretch.

MPLS provides modularization by providing abstraction with labels. MPLS leverages the concept of predetermined “labels” to route traffic instead of relying solely on the ultimate source and destination addresses. This is done by appending a short bit sequence to the packet, known as forwarding equivalence class (FEC) or class of service (CoS).

Enhanced Scalability and Flexibility:

One of the primary benefits of modularization virtualization is its ability to enhance scalability and flexibility. Organizations can quickly scale their infrastructure up or down by virtualizing individual modules based on demand. This flexibility allows businesses to adapt rapidly to changing market conditions and optimize resource allocation.

Improved Fault Isolation and Resilience:

Modularization virtualization also improves fault isolation and resilience. Since each module operates independently, a failure or issue in one module does not impact the entire system. This isolation ensures that critical functions remain unaffected, enhancing the overall reliability and uptime of the system.

Simplified Development and Maintenance:

With modularization, virtualization, development, and maintenance become more manageable and efficient. Each module can be developed and tested independently, enabling faster deployment and reducing the risk of errors. Additionally, updates or changes to a specific module can be implemented without disrupting the entire system, minimizing downtime and reducing maintenance efforts.